Pushing the docs to dev/ for branch: main, commit ee32699b4c2ecf986786de57ebb78dcf602bca8e

Author: sklearn-ci

dev/_downloads/07fcc19ba03226cd3d83d4e40ec44385/auto_examples_python.zip

@@ -15,7 +15,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "\n# Pipeline Anova SVM\n\nSimple usage of Pipeline that runs successively a univariate\nfeature selection with anova and then a SVM of the selected features.\n\nUsing a sub-pipeline, the fitted coefficients can be mapped back into\nthe original feature space.\n"
+        "\n# Pipeline ANOVA SVM\n\nThis example shows how a feature selection can be easily integrated within\na machine learning pipeline.\n\nWe also show that you can easily introspect part of the pipeline.\n"
       ]
     },
     {
@@ -26,7 +26,104 @@
       },
       "outputs": [],
       "source": [
-        "from sklearn import svm\nfrom sklearn.datasets import make_classification\nfrom sklearn.feature_selection import SelectKBest, f_classif\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import classification_report\n\nprint(__doc__)\n\n# import some data to play with\nX, y = make_classification(\n    n_features=20, n_informative=3, n_redundant=0, n_classes=4,\n    n_clusters_per_class=2)\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)\n\n# ANOVA SVM-C\n# 1) anova filter, take 3 best ranked features\nanova_filter = SelectKBest(f_classif, k=3)\n# 2) svm\nclf = svm.LinearSVC()\n\nanova_svm = make_pipeline(anova_filter, clf)\nanova_svm.fit(X_train, y_train)\ny_pred = anova_svm.predict(X_test)\nprint(classification_report(y_test, y_pred))\n\ncoef = anova_svm[:-1].inverse_transform(anova_svm['linearsvc'].coef_)\nprint(coef)"
+        "print(__doc__)\nfrom sklearn import set_config\nset_config(display='diagram')"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "We will start by generating a binary classification dataset. Subsequently, we\nwill divide the dataset into two subsets.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "from sklearn.datasets import make_classification\nfrom sklearn.model_selection import train_test_split\n\nX, y = make_classification(\n    n_features=20, n_informative=3, n_redundant=0, n_classes=2,\n    n_clusters_per_class=2, random_state=42)\nX_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "A common mistake done with feature selection is to search a subset of\ndiscriminative features on the full dataset instead of only using the\ntraining set. The usage of scikit-learn :func:`~sklearn.pipeline.Pipeline`\nprevents to make such mistake.\n\nHere, we will demonstrate how to build a pipeline where the first step will\nbe the feature selection.\n\nWhen calling `fit` on the training data, a subset of feature will be selected\nand the index of these selected features will be stored. The feature selector\nwill subsequently reduce the number of feature and pass this subset to the\nclassifier which will be trained.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "from sklearn.feature_selection import SelectKBest, f_classif\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.svm import LinearSVC\n\nanova_filter = SelectKBest(f_classif, k=3)\nclf = LinearSVC()\nanova_svm = make_pipeline(anova_filter, clf)\nanova_svm.fit(X_train, y_train)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Once the training accomplished, we can predict on new unseen samples. In this\ncase, the feature selector will only select the most discriminative features\nbased on the information stored during training. Then, the data will be\npassed to the classifier which will make the prediction.\n\nHere, we report the final metrics via a classification report.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "from sklearn.metrics import classification_report\n\ny_pred = anova_svm.predict(X_test)\nprint(classification_report(y_test, y_pred))"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Be aware that you can inspect a step in the pipeline. For instance, we might\nbe interested about the parameters of the classifier. Since we selected\nthree features, we expect to have three coefficients.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "anova_svm[-1].coef_"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "However, we do not know which features where selected from the original\ndataset. We could proceed by several manner. Here, we will inverse the\ntransformation of these coefficients to get information about the original\nspace.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "anova_svm[:-1].inverse_transform(anova_svm[-1].coef_)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "We can see that the first three features where the selected features by\nthe first step.\n\n"
       ]
     }
   ],