Pushing the docs to dev/ for branch: main, commit 266d2a2b8d5be2fb074415e22f7411484fd35a46

Author: sklearn-ci

dev/_downloads/07fcc19ba03226cd3d83d4e40ec44385/auto_examples_python.zip

@@ -15,7 +15,32 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "\n# Detection error tradeoff (DET) curve\n\nIn this example, we compare receiver operating characteristic (ROC) and\ndetection error tradeoff (DET) curves for different classification algorithms\nfor the same classification task.\n\nDET curves are commonly plotted in normal deviate scale.\nTo achieve this the DET display transforms the error rates as returned by the\n:func:`~sklearn.metrics.det_curve` and the axis scale using\n:func:`scipy.stats.norm`.\n\nThe point of this example is to demonstrate two properties of DET curves,\nnamely:\n\n1. It might be easier to visually assess the overall performance of different\n   classification algorithms using DET curves over ROC curves.\n   Due to the linear scale used for plotting ROC curves, different classifiers\n   usually only differ in the top left corner of the graph and appear similar\n   for a large part of the plot. On the other hand, because DET curves\n   represent straight lines in normal deviate scale. As such, they tend to be\n   distinguishable as a whole and the area of interest spans a large part of\n   the plot.\n2. DET curves give the user direct feedback of the detection error tradeoff to\n   aid in operating point analysis.\n   The user can deduct directly from the DET-curve plot at which rate\n   false-negative error rate will improve when willing to accept an increase in\n   false-positive error rate (or vice-versa).\n\nThe plots in this example compare ROC curves on the left side to corresponding\nDET curves on the right.\nThere is no particular reason why these classifiers have been chosen for the\nexample plot over other classifiers available in scikit-learn.\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>- See :func:`sklearn.metrics.roc_curve` for further information about ROC\n      curves.\n\n    - See :func:`sklearn.metrics.det_curve` for further information about\n      DET curves.\n\n    - This example is loosely based on\n      `sphx_glr_auto_examples_classification_plot_classifier_comparison.py`\n      example.</p></div>\n"
+        "\n# Detection error tradeoff (DET) curve\n\nIn this example, we compare two binary classification multi-threshold metrics:\nthe Receiver Operating Characteristic (ROC) and the Detection Error Tradeoff\n(DET). For such purpose, we evaluate two different classifiers for the same\nclassification task.\n\nROC curves feature true positive rate (TPR) on the Y axis, and false positive\nrate (FPR) on the X axis. This means that the top left corner of the plot is the\n\"ideal\" point - a FPR of zero, and a TPR of one.\n\nDET curves are a variation of ROC curves where False Negative Rate (FNR) is\nplotted on the y-axis instead of the TPR. In this case the origin (bottom left\ncorner) is the \"ideal\" point.\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>- See :func:`sklearn.metrics.roc_curve` for further information about ROC\n      curves.\n\n    - See :func:`sklearn.metrics.det_curve` for further information about\n      DET curves.\n\n    - This example is loosely based on\n      `sphx_glr_auto_examples_classification_plot_classifier_comparison.py`\n      example.\n\n    - See `sphx_glr_auto_examples_model_selection_plot_roc_crossval.py` for\n      an example estimating the variance of the ROC curves and ROC-AUC.</p></div>\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Generate synthetic data\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "from sklearn.datasets import make_classification\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import StandardScaler\n\nX, y = make_classification(\n    n_samples=1_000,\n    n_features=2,\n    n_redundant=0,\n    n_informative=2,\n    random_state=1,\n    n_clusters_per_class=1,\n)\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=0)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Define the classifiers\n\nHere we define two different classifiers. The goal is to visualy compare their\nstatistical performance across thresholds using the ROC and DET curves. There\nis no particular reason why these classifiers are chosen other classifiers\navailable in scikit-learn.\n\n"
       ]
     },
     {
@@ -26,7 +51,32 @@
       },
       "outputs": [],
       "source": [
-        "import matplotlib.pyplot as plt\n\nfrom sklearn.datasets import make_classification\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.metrics import DetCurveDisplay, RocCurveDisplay\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.svm import LinearSVC\n\nN_SAMPLES = 1000\n\nclassifiers = {\n    \"Linear SVM\": make_pipeline(StandardScaler(), LinearSVC(C=0.025)),\n    \"Random Forest\": RandomForestClassifier(\n        max_depth=5, n_estimators=10, max_features=1\n    ),\n}\n\nX, y = make_classification(\n    n_samples=N_SAMPLES,\n    n_features=2,\n    n_redundant=0,\n    n_informative=2,\n    random_state=1,\n    n_clusters_per_class=1,\n)\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=0)\n\n# prepare plots\nfig, [ax_roc, ax_det] = plt.subplots(1, 2, figsize=(11, 5))\n\nfor name, clf in classifiers.items():\n    clf.fit(X_train, y_train)\n\n    RocCurveDisplay.from_estimator(clf, X_test, y_test, ax=ax_roc, name=name)\n    DetCurveDisplay.from_estimator(clf, X_test, y_test, ax=ax_det, name=name)\n\nax_roc.set_title(\"Receiver Operating Characteristic (ROC) curves\")\nax_det.set_title(\"Detection Error Tradeoff (DET) curves\")\n\nax_roc.grid(linestyle=\"--\")\nax_det.grid(linestyle=\"--\")\n\nplt.legend()\nplt.show()"
+        "from sklearn.ensemble import RandomForestClassifier\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.svm import LinearSVC\n\nclassifiers = {\n    \"Linear SVM\": make_pipeline(StandardScaler(), LinearSVC(C=0.025)),\n    \"Random Forest\": RandomForestClassifier(\n        max_depth=5, n_estimators=10, max_features=1\n    ),\n}"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Plot ROC and DET curves\n\nDET curves are commonly plotted in normal deviate scale. To achieve this the\nDET display transforms the error rates as returned by the\n:func:`~sklearn.metrics.det_curve` and the axis scale using\n:func:`scipy.stats.norm`.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "import matplotlib.pyplot as plt\nfrom sklearn.metrics import DetCurveDisplay, RocCurveDisplay\n\nfig, [ax_roc, ax_det] = plt.subplots(1, 2, figsize=(11, 5))\n\nfor name, clf in classifiers.items():\n    clf.fit(X_train, y_train)\n\n    RocCurveDisplay.from_estimator(clf, X_test, y_test, ax=ax_roc, name=name)\n    DetCurveDisplay.from_estimator(clf, X_test, y_test, ax=ax_det, name=name)\n\nax_roc.set_title(\"Receiver Operating Characteristic (ROC) curves\")\nax_det.set_title(\"Detection Error Tradeoff (DET) curves\")\n\nax_roc.grid(linestyle=\"--\")\nax_det.grid(linestyle=\"--\")\n\nplt.legend()\nplt.show()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Notice that it is easier to visually assess the overall performance of\ndifferent classification algorithms using DET curves than using ROC curves. As\nROC curves are plot in a linear scale, different classifiers usually appear\nsimilar for a large part of the plot and differ the most in the top left\ncorner of the graph. On the other hand, because DET curves represent straight\nlines in normal deviate scale, they tend to be distinguishable as a whole and\nthe area of interest spans a large part of the plot.\n\nDET curves give direct feedback of the detection error tradeoff to aid in\noperating point analysis. The user can then decide the FNR they are willing to\naccept at the expense of the FPR (or vice-versa).\n\n"
       ]
     }
   ],