Pushing the docs to dev/ for branch: master, commit 4f710cdd088aa8851e8b049e4faafa03767fda10

Author: sklearn-ci

dev/_downloads/auto_examples_jupyter.zip

@@ -0,0 +1,187 @@
+{
+  "cells": [
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "%matplotlib inline"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n# Effect of transforming the targets in regression model\n\n\nIn this example, we give an overview of the\n:class:`sklearn.preprocessing.TransformedTargetRegressor`. Two examples\nillustrate the benefit of transforming the targets before learning a linear\nregression model. The first example uses synthetic data while the second\nexample is based on the Boston housing data set.\n\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "# Author: Guillaume Lemaitre <[email protected]>\n# License: BSD 3 clause\n\nfrom __future__ import print_function, division\n\nimport numpy as np\nimport matplotlib.pyplot as plt\n\nprint(__doc__)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Synthetic example\n##############################################################################\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "from sklearn.datasets import make_regression\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import RidgeCV\nfrom sklearn.preprocessing import TransformedTargetRegressor\nfrom sklearn.metrics import median_absolute_error, r2_score"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "A synthetic random regression problem is generated. The targets ``y`` are\nmodified by: (i) translating all targets such that all entries are\nnon-negative and (ii) applying an exponential function to obtain non-linear\ntargets which cannot be fitted using a simple linear model.\n\nTherefore, a logarithmic and an exponential function will be used to\ntransform the targets before training a linear regression model and using it\nfor prediction.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "def log_transform(x):\n    return np.log(x + 1)\n\n\ndef exp_transform(x):\n    return np.exp(x) - 1\n\n\nX, y = make_regression(n_samples=10000, noise=100, random_state=0)\ny = np.exp((y + abs(y.min())) / 200)\ny_trans = log_transform(y)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "The following illustrate the probability density functions of the target\nbefore and after applying the logarithmic functions.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "f, (ax0, ax1) = plt.subplots(1, 2)\n\nax0.hist(y, bins='auto', normed=True)\nax0.set_xlim([0, 2000])\nax0.set_ylabel('Probability')\nax0.set_xlabel('Target')\nax0.set_title('Target distribution')\n\nax1.hist(y_trans, bins='auto', normed=True)\nax1.set_ylabel('Probability')\nax1.set_xlabel('Target')\nax1.set_title('Transformed target distribution')\n\nf.suptitle(\"Synthetic data\", y=0.035)\nf.tight_layout(rect=[0.05, 0.05, 0.95, 0.95])\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "At first, a linear model will be applied on the original targets. Due to the\nnon-linearity, the model trained will not be precise during the\nprediction. Subsequently, a logarithmic function is used to linearize the\ntargets, allowing better prediction even with a similar linear model as\nreported by the median absolute error (MAE).\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "f, (ax0, ax1) = plt.subplots(1, 2, sharey=True)\n\nregr = RidgeCV()\nregr.fit(X_train, y_train)\ny_pred = regr.predict(X_test)\n\nax0.scatter(y_test, y_pred)\nax0.plot([0, 2000], [0, 2000], '--k')\nax0.set_ylabel('Target predicted')\nax0.set_xlabel('True Target')\nax0.set_title('Ridge regression \\n without target transformation')\nax0.text(100, 1750, r'$R^2$=%.2f, MAE=%.2f' % (\n    r2_score(y_test, y_pred), median_absolute_error(y_test, y_pred)))\nax0.set_xlim([0, 2000])\nax0.set_ylim([0, 2000])\n\nregr_trans = TransformedTargetRegressor(regressor=RidgeCV(),\n                                        func=log_transform,\n                                        inverse_func=exp_transform)\nregr_trans.fit(X_train, y_train)\ny_pred = regr_trans.predict(X_test)\n\nax1.scatter(y_test, y_pred)\nax1.plot([0, 2000], [0, 2000], '--k')\nax1.set_ylabel('Target predicted')\nax1.set_xlabel('True Target')\nax1.set_title('Ridge regression \\n with target transformation')\nax1.text(100, 1750, r'$R^2$=%.2f, MAE=%.2f' % (\n    r2_score(y_test, y_pred), median_absolute_error(y_test, y_pred)))\nax1.set_xlim([0, 2000])\nax1.set_ylim([0, 2000])\n\nf.suptitle(\"Synthetic data\", y=0.035)\nf.tight_layout(rect=[0.05, 0.05, 0.95, 0.95])"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Real-world data set\n##############################################################################\n\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "In a similar manner, the boston housing data set is used to show the impact\nof transforming the targets before learning a model. In this example, the\ntargets to be predicted corresponds to the weighted distances to the five\nBoston employment centers.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "from sklearn.datasets import load_boston\nfrom sklearn.preprocessing import QuantileTransformer, quantile_transform\n\ndataset = load_boston()\ntarget = np.array(dataset.feature_names) == \"DIS\"\nX = dataset.data[:, np.logical_not(target)]\ny = dataset.data[:, target].squeeze()\ny_trans = quantile_transform(dataset.data[:, target],\n                             output_distribution='normal').squeeze()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "A :class:`sklearn.preprocessing.QuantileTransformer` is used such that the\ntargets follows a normal distribution before applying a\n:class:`sklearn.linear_model.RidgeCV` model.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "f, (ax0, ax1) = plt.subplots(1, 2)\n\nax0.hist(y, bins='auto', normed=True)\nax0.set_ylabel('Probability')\nax0.set_xlabel('Target')\nax0.set_title('Target distribution')\n\nax1.hist(y_trans, bins='auto', normed=True)\nax1.set_ylabel('Probability')\nax1.set_xlabel('Target')\nax1.set_title('Transformed target distribution')\n\nf.suptitle(\"Boston housing data: distance to employment centers\", y=0.035)\nf.tight_layout(rect=[0.05, 0.05, 0.95, 0.95])\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "The effect of the transformer is weaker than on the synthetic data. However,\nthe transform induces a decrease of the MAE.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "f, (ax0, ax1) = plt.subplots(1, 2, sharey=True)\n\nregr = RidgeCV()\nregr.fit(X_train, y_train)\ny_pred = regr.predict(X_test)\n\nax0.scatter(y_test, y_pred)\nax0.plot([0, 10], [0, 10], '--k')\nax0.set_ylabel('Target predicted')\nax0.set_xlabel('True Target')\nax0.set_title('Ridge regression \\n without target transformation')\nax0.text(1, 9, r'$R^2$=%.2f, MAE=%.2f' % (\n    r2_score(y_test, y_pred), median_absolute_error(y_test, y_pred)))\nax0.set_xlim([0, 10])\nax0.set_ylim([0, 10])\n\nregr_trans = TransformedTargetRegressor(\n    regressor=RidgeCV(),\n    transformer=QuantileTransformer(output_distribution='normal'))\nregr_trans.fit(X_train, y_train)\ny_pred = regr_trans.predict(X_test)\n\nax1.scatter(y_test, y_pred)\nax1.plot([0, 10], [0, 10], '--k')\nax1.set_ylabel('Target predicted')\nax1.set_xlabel('True Target')\nax1.set_title('Ridge regression \\n with target transformation')\nax1.text(1, 9, r'$R^2$=%.2f, MAE=%.2f' % (\n    r2_score(y_test, y_pred), median_absolute_error(y_test, y_pred)))\nax1.set_xlim([0, 10])\nax1.set_ylim([0, 10])\n\nf.suptitle(\"Boston housing data: distance to employment centers\", y=0.035)\nf.tight_layout(rect=[0.05, 0.05, 0.95, 0.95])\n\nplt.show()"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.6.3"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

dev/_downloads/auto_examples_python.zip

@@ -0,0 +1,205 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""
+======================================================
+Effect of transforming the targets in regression model
+======================================================
+
+In this example, we give an overview of the
+:class:`sklearn.preprocessing.TransformedTargetRegressor`. Two examples
+illustrate the benefit of transforming the targets before learning a linear
+regression model. The first example uses synthetic data while the second
+example is based on the Boston housing data set.
+
+"""
+
+# Author: Guillaume Lemaitre <[email protected]>
+# License: BSD 3 clause
+
+from __future__ import print_function, division
+
+import numpy as np
+import matplotlib.pyplot as plt
+
+print(__doc__)
+
+###############################################################################
+# Synthetic example
+###############################################################################
+
+from sklearn.datasets import make_regression
+from sklearn.model_selection import train_test_split
+from sklearn.linear_model import RidgeCV
+from sklearn.preprocessing import TransformedTargetRegressor
+from sklearn.metrics import median_absolute_error, r2_score
+
+###############################################################################
+# A synthetic random regression problem is generated. The targets ``y`` are
+# modified by: (i) translating all targets such that all entries are
+# non-negative and (ii) applying an exponential function to obtain non-linear
+# targets which cannot be fitted using a simple linear model.
+#
+# Therefore, a logarithmic and an exponential function will be used to
+# transform the targets before training a linear regression model and using it
+# for prediction.
+
+
+def log_transform(x):
+    return np.log(x + 1)
+
+
+def exp_transform(x):
+    return np.exp(x) - 1
+
+
+X, y = make_regression(n_samples=10000, noise=100, random_state=0)
+y = np.exp((y + abs(y.min())) / 200)
+y_trans = log_transform(y)
+
+###############################################################################
+# The following illustrate the probability density functions of the target
+# before and after applying the logarithmic functions.
+
+f, (ax0, ax1) = plt.subplots(1, 2)
+
+ax0.hist(y, bins='auto', normed=True)
+ax0.set_xlim([0, 2000])
+ax0.set_ylabel('Probability')
+ax0.set_xlabel('Target')
+ax0.set_title('Target distribution')
+
+ax1.hist(y_trans, bins='auto', normed=True)
+ax1.set_ylabel('Probability')
+ax1.set_xlabel('Target')
+ax1.set_title('Transformed target distribution')
+
+f.suptitle("Synthetic data", y=0.035)
+f.tight_layout(rect=[0.05, 0.05, 0.95, 0.95])
+
+X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
+
+###############################################################################
+# At first, a linear model will be applied on the original targets. Due to the
+# non-linearity, the model trained will not be precise during the
+# prediction. Subsequently, a logarithmic function is used to linearize the
+# targets, allowing better prediction even with a similar linear model as
+# reported by the median absolute error (MAE).
+
+f, (ax0, ax1) = plt.subplots(1, 2, sharey=True)
+
+regr = RidgeCV()
+regr.fit(X_train, y_train)
+y_pred = regr.predict(X_test)
+
+ax0.scatter(y_test, y_pred)
+ax0.plot([0, 2000], [0, 2000], '--k')
+ax0.set_ylabel('Target predicted')
+ax0.set_xlabel('True Target')
+ax0.set_title('Ridge regression \n without target transformation')
+ax0.text(100, 1750, r'$R^2$=%.2f, MAE=%.2f' % (
+    r2_score(y_test, y_pred), median_absolute_error(y_test, y_pred)))
+ax0.set_xlim([0, 2000])
+ax0.set_ylim([0, 2000])
+
+regr_trans = TransformedTargetRegressor(regressor=RidgeCV(),
+                                        func=log_transform,
+                                        inverse_func=exp_transform)
+regr_trans.fit(X_train, y_train)
+y_pred = regr_trans.predict(X_test)
+
+ax1.scatter(y_test, y_pred)
+ax1.plot([0, 2000], [0, 2000], '--k')
+ax1.set_ylabel('Target predicted')
+ax1.set_xlabel('True Target')
+ax1.set_title('Ridge regression \n with target transformation')
+ax1.text(100, 1750, r'$R^2$=%.2f, MAE=%.2f' % (
+    r2_score(y_test, y_pred), median_absolute_error(y_test, y_pred)))
+ax1.set_xlim([0, 2000])
+ax1.set_ylim([0, 2000])
+
+f.suptitle("Synthetic data", y=0.035)
+f.tight_layout(rect=[0.05, 0.05, 0.95, 0.95])
+
+###############################################################################
+# Real-world data set
+###############################################################################
+
+###############################################################################
+# In a similar manner, the boston housing data set is used to show the impact
+# of transforming the targets before learning a model. In this example, the
+# targets to be predicted corresponds to the weighted distances to the five
+# Boston employment centers.
+
+from sklearn.datasets import load_boston
+from sklearn.preprocessing import QuantileTransformer, quantile_transform
+
+dataset = load_boston()
+target = np.array(dataset.feature_names) == "DIS"
+X = dataset.data[:, np.logical_not(target)]
+y = dataset.data[:, target].squeeze()
+y_trans = quantile_transform(dataset.data[:, target],
+                             output_distribution='normal').squeeze()
+
+###############################################################################
+# A :class:`sklearn.preprocessing.QuantileTransformer` is used such that the
+# targets follows a normal distribution before applying a
+# :class:`sklearn.linear_model.RidgeCV` model.
+
+f, (ax0, ax1) = plt.subplots(1, 2)
+
+ax0.hist(y, bins='auto', normed=True)
+ax0.set_ylabel('Probability')
+ax0.set_xlabel('Target')
+ax0.set_title('Target distribution')
+
+ax1.hist(y_trans, bins='auto', normed=True)
+ax1.set_ylabel('Probability')
+ax1.set_xlabel('Target')
+ax1.set_title('Transformed target distribution')
+
+f.suptitle("Boston housing data: distance to employment centers", y=0.035)
+f.tight_layout(rect=[0.05, 0.05, 0.95, 0.95])
+
+X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)
+
+###############################################################################
+# The effect of the transformer is weaker than on the synthetic data. However,
+# the transform induces a decrease of the MAE.
+
+f, (ax0, ax1) = plt.subplots(1, 2, sharey=True)
+
+regr = RidgeCV()
+regr.fit(X_train, y_train)
+y_pred = regr.predict(X_test)
+
+ax0.scatter(y_test, y_pred)
+ax0.plot([0, 10], [0, 10], '--k')
+ax0.set_ylabel('Target predicted')
+ax0.set_xlabel('True Target')
+ax0.set_title('Ridge regression \n without target transformation')
+ax0.text(1, 9, r'$R^2$=%.2f, MAE=%.2f' % (
+    r2_score(y_test, y_pred), median_absolute_error(y_test, y_pred)))
+ax0.set_xlim([0, 10])
+ax0.set_ylim([0, 10])
+
+regr_trans = TransformedTargetRegressor(
+    regressor=RidgeCV(),
+    transformer=QuantileTransformer(output_distribution='normal'))
+regr_trans.fit(X_train, y_train)
+y_pred = regr_trans.predict(X_test)
+
+ax1.scatter(y_test, y_pred)
+ax1.plot([0, 10], [0, 10], '--k')
+ax1.set_ylabel('Target predicted')
+ax1.set_xlabel('True Target')
+ax1.set_title('Ridge regression \n with target transformation')
+ax1.text(1, 9, r'$R^2$=%.2f, MAE=%.2f' % (
+    r2_score(y_test, y_pred), median_absolute_error(y_test, y_pred)))
+ax1.set_xlim([0, 10])
+ax1.set_ylim([0, 10])
+
+f.suptitle("Boston housing data: distance to employment centers", y=0.035)
+f.tight_layout(rect=[0.05, 0.05, 0.95, 0.95])
+
+plt.show()