Pushing the docs to dev/ for branch: main, commit 5ceb8a6a031ddff26a7ede413db1b53edb64166a

Author: sklearn-ci

dev/_downloads/07fcc19ba03226cd3d83d4e40ec44385/auto_examples_python.zip

@@ -137,7 +137,61 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "The two-way partial dependence plot shows the dependence of median house\nprice on joint values of house age and average occupants per household. We\ncan clearly see an interaction between the two features: for an average\noccupancy greater than two, the house price is nearly independent of the\nhouse age, whereas for values less than two there is a strong dependence on\nage.\n\n## 3D interaction plots\n\nLet's make the same partial dependence plot for the 2 features interaction,\nthis time in 3 dimensions.\n\n"
+        "The two-way partial dependence plot shows the dependence of median house\nprice on joint values of house age and average occupants per household. We\ncan clearly see an interaction between the two features: for an average\noccupancy greater than two, the house price is nearly independent of the\nhouse age, whereas for values less than two there is a strong dependence on\nage.\n\n### Interaction constraints\n\nThe histogram gradient boosters have an interesting option to constrain\npossible interactions among features. In the following, we do not allow any\ninteractions and thus render the model as a version of a tree-based boosted\ngeneralized additive model (GAM). This makes the model more interpretable\nas the effect of each feature can be investigated independently of all others.\n\nWe train the :class:`~sklearn.ensemble.HistGradientBoostingRegressor` again,\nnow with `interaction_cst`, where we  pass for each feature a list containing\nonly its own index, e.g. `[[0], [1], [2], ..]`.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "print(\"Training interaction constraint HistGradientBoostingRegressor...\")\ntic = time()\nest_no_interactions = HistGradientBoostingRegressor(\n    interaction_cst=[[i] for i in range(X_train.shape[1])]\n)\nest_no_interactions.fit(X_train, y_train)\nprint(f\"done in {time() - tic:.3f}s\")"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "The easiest way to show the effect of forbidden interactions is again the\nICE plots.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "print(\"Computing partial dependence plots...\")\ntic = time()\ndisplay = PartialDependenceDisplay.from_estimator(\n    est_no_interactions,\n    X_train,\n    [\"MedInc\", \"AveOccup\", \"HouseAge\", \"AveRooms\"],\n    kind=\"both\",\n    subsample=50,\n    n_jobs=3,\n    grid_resolution=20,\n    random_state=0,\n    ice_lines_kw={\"color\": \"tab:blue\", \"alpha\": 0.2, \"linewidth\": 0.5},\n    pd_line_kw={\"color\": \"tab:orange\", \"linestyle\": \"--\"},\n)\n\nprint(f\"done in {time() - tic:.3f}s\")\ndisplay.figure_.suptitle(\n    \"Partial dependence of house value with Gradient Boosting\\n\"\n    \"and no interactions allowed\"\n)\ndisplay.figure_.subplots_adjust(wspace=0.4, hspace=0.3)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "All 4 plots have parallel ICE lines meaning there is no interaction in the\nmodel.\nLet us also have a look at the corresponding 2D-plot.\n\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "collapsed": false
+      },
+      "outputs": [],
+      "source": [
+        "print(\"Computing partial dependence plots...\")\ntic = time()\n_, ax = plt.subplots(ncols=3, figsize=(9, 4))\ndisplay = PartialDependenceDisplay.from_estimator(\n    est_no_interactions,\n    X_train,\n    [\"AveOccup\", \"HouseAge\", (\"AveOccup\", \"HouseAge\")],\n    kind=\"average\",\n    n_jobs=3,\n    grid_resolution=20,\n    ax=ax,\n)\nprint(f\"done in {time() - tic:.3f}s\")\ndisplay.figure_.suptitle(\n    \"Partial dependence of house value with Gradient Boosting\\n\"\n    \"and no interactions allowed\"\n)\ndisplay.figure_.subplots_adjust(wspace=0.4, hspace=0.3)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Although the 2D-plot shows much less interaction compared with the 2D-plot\nfrom above, it is much harder to come to the conclusion that there is no\ninteraction at all. This might be a cause of the discrete predictions of\ntrees in combination with numerically precision of partial dependence.\nWe also observe that the univariate dependence plots have slightly changed\nas the model tries to compensate for the forbidden interactions.\n\n## 3D interaction plots\n\nLet's make the same partial dependence plot for the 2 features interaction,\nthis time in 3 dimensions.\n\n"
       ]
     },
     {

dev/_downloads/21b82d82985712b5de6347f382c77c86/plot_partial_dependence.ipynb

@@ -255,6 +255,85 @@
 # house age, whereas for values less than two there is a strong dependence on
 # age.
 #
+# Interaction constraints
+# .......................
+#
+# The histogram gradient boosters have an interesting option to constrain
+# possible interactions among features. In the following, we do not allow any
+# interactions and thus render the model as a version of a tree-based boosted
+# generalized additive model (GAM). This makes the model more interpretable
+# as the effect of each feature can be investigated independently of all others.
+#
+# We train the :class:`~sklearn.ensemble.HistGradientBoostingRegressor` again,
+# now with `interaction_cst`, where we  pass for each feature a list containing
+# only its own index, e.g. `[[0], [1], [2], ..]`.
+
+print("Training interaction constraint HistGradientBoostingRegressor...")
+tic = time()
+est_no_interactions = HistGradientBoostingRegressor(
+    interaction_cst=[[i] for i in range(X_train.shape[1])]
+)
+est_no_interactions.fit(X_train, y_train)
+print(f"done in {time() - tic:.3f}s")
+
+# %%
+# The easiest way to show the effect of forbidden interactions is again the
+# ICE plots.
+
+print("Computing partial dependence plots...")
+tic = time()
+display = PartialDependenceDisplay.from_estimator(
+    est_no_interactions,
+    X_train,
+    ["MedInc", "AveOccup", "HouseAge", "AveRooms"],
+    kind="both",
+    subsample=50,
+    n_jobs=3,
+    grid_resolution=20,
+    random_state=0,
+    ice_lines_kw={"color": "tab:blue", "alpha": 0.2, "linewidth": 0.5},
+    pd_line_kw={"color": "tab:orange", "linestyle": "--"},
+)
+
+print(f"done in {time() - tic:.3f}s")
+display.figure_.suptitle(
+    "Partial dependence of house value with Gradient Boosting\n"
+    "and no interactions allowed"
+)
+display.figure_.subplots_adjust(wspace=0.4, hspace=0.3)
+
+# %%
+# All 4 plots have parallel ICE lines meaning there is no interaction in the
+# model.
+# Let us also have a look at the corresponding 2D-plot.
+
+print("Computing partial dependence plots...")
+tic = time()
+_, ax = plt.subplots(ncols=3, figsize=(9, 4))
+display = PartialDependenceDisplay.from_estimator(
+    est_no_interactions,
+    X_train,
+    ["AveOccup", "HouseAge", ("AveOccup", "HouseAge")],
+    kind="average",
+    n_jobs=3,
+    grid_resolution=20,
+    ax=ax,
+)
+print(f"done in {time() - tic:.3f}s")
+display.figure_.suptitle(
+    "Partial dependence of house value with Gradient Boosting\n"
+    "and no interactions allowed"
+)
+display.figure_.subplots_adjust(wspace=0.4, hspace=0.3)
+
+# %%
+# Although the 2D-plot shows much less interaction compared with the 2D-plot
+# from above, it is much harder to come to the conclusion that there is no
+# interaction at all. This might be a cause of the discrete predictions of
+# trees in combination with numerically precision of partial dependence.
+# We also observe that the univariate dependence plots have slightly changed
+# as the model tries to compensate for the forbidden interactions.
+#
 # 3D interaction plots
 # --------------------
 #