diff --git a/README.md b/README.md index b97bd9f0e..a57c07e7c 100644 --- a/README.md +++ b/README.md @@ -1,35 +1,76 @@ -# Companion Jupyter notebooks for the book "Deep Learning with Python" +# Companion notebooks for Deep Learning with Python -This repository contains Jupyter notebooks implementing the code samples found in the book [Deep Learning with Python, 2nd Edition (Manning Publications)](https://www.manning.com/books/deep-learning-with-python-second-edition?a_aid=keras&a_bid=76564dff). +This repository contains Jupyter notebooks implementing the code samples found in the book [Deep Learning with Python, third edition (2025)](https://www.manning.com/books/deep-learning-with-python-third-edition?a_aid=keras&a_bid=76564dff) +by Francois Chollet and Matthew Watson. In addition, you will also find the legacy notebooks for the [second edition (2021)](https://www.manning.com/books/deep-learning-with-python-second-edition?a_aid=keras&a_bid=76564dff) +and the [first edition (2017)](https://www.manning.com/books/deep-learning-with-python?a_aid=keras&a_bid=76564dff). For readability, these notebooks only contain runnable code blocks and section titles, and omit everything else in the book: text paragraphs, figures, and pseudocode. **If you want to be able to follow what's going on, I recommend reading the notebooks side by side with your copy of the book.** -These notebooks use TensorFlow 2.6. +## Running the code + +We recommend running these notebooks on [Colab](https://colab.google), which +provides a hosted runtime with all the dependencies you will need. You can also, +run these notebooks locally, either by setting up your own Jupyter environment, +or using Colab's instructions for +[running locally](https://research.google.com/colaboratory/local-runtimes.html). + +By default, all notebooks will run on Colab's free tier GPU runtime, which +is sufficient to run all code in this book. Chapter 8-18 chapters will benefit +from a faster GPU if you have a Colab Pro subscription. You can change your +runtime type using **Runtime -> Change runtime type** in Colab's dropdown menus. + +## Choosing a backend + +The code for third edition is written using Keras 3. As such, it can be run with +JAX, TensorFlow or PyTorch as a backend. To set the backend, update the backend +in the cell at the top of the colab that looks like this: + +```python +import os +os.environ["KERAS_BACKEND"] = "jax" +``` + +This must be done only once per session before importing Keras. If you are +in the middle running a notebook, you will need to restart the notebook session +and rerun all relevant notebook cells. This can be done in using +**Runtime -> Restart Session** in Colab's dropdown menus. + +## Using Kaggle data + +This book uses datasets and model weights provided by Kaggle, an online Machine +Learning community and platform. You will need to create a Kaggle login to run +Kaggle code in this book; instructions are given in Chapter 8. + +For chapters that need Kaggle data, you can login to Kaggle once per session +when you hit the notebook cell with `kagglehub.login()`. Alternately, +you can set up your Kaggle login information once as Colab secrets: + + * Go to https://www.kaggle.com/ and sign in. + * Go to https://www.kaggle.com/settings and generate a Kaggle API key. + * Open the secrets tab in Colab by clicking the key icon on the left. + * Add two secrets, `KAGGLE_USERNAME` and `KAGGLE_KEY` with the username and key + you just created. + +Following this approach you will only need to copy your Kaggle secret key once, +though you will need to allow each notebook to access your secrets when running +the relevant Kaggle code. ## Table of contents * [Chapter 2: The mathematical building blocks of neural networks](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter02_mathematical-building-blocks.ipynb) -* [Chapter 3: Introduction to Keras and TensorFlow](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter03_introduction-to-keras-and-tf.ipynb) -* [Chapter 4: Getting started with neural networks: classification and regression](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter04_getting-started-with-neural-networks.ipynb) +* [Chapter 3: Introduction to TensorFlow, PyTorch, JAX, and Keras](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter03_introduction-to-ml-frameworks.ipynb) +* [Chapter 4: Classification and regression](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter04_classification-and-regression.ipynb) * [Chapter 5: Fundamentals of machine learning](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter05_fundamentals-of-ml.ipynb) -* [Chapter 7: Working with Keras: a deep dive](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter07_working-with-keras.ipynb) -* [Chapter 8: Introduction to deep learning for computer vision](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter08_intro-to-dl-for-computer-vision.ipynb) -* Chapter 9: Advanced deep learning for computer vision - - [Part 1: Image segmentation](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter09_part01_image-segmentation.ipynb) - - [Part 2: Modern convnet architecture patterns](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter09_part02_modern-convnet-architecture-patterns.ipynb) - - [Part 3: Interpreting what convnets learn](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter09_part03_interpreting-what-convnets-learn.ipynb) -* [Chapter 10: Deep learning for timeseries](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter10_dl-for-timeseries.ipynb) -* Chapter 11: Deep learning for text - - [Part 1: Introduction](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter11_part01_introduction.ipynb) - - [Part 2: Sequence models](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter11_part02_sequence-models.ipynb) - - [Part 3: Transformer](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter11_part03_transformer.ipynb) - - [Part 4: Sequence-to-sequence learning](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter11_part04_sequence-to-sequence-learning.ipynb) -* Chapter 12: Generative deep learning - - [Part 1: Text generation](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter12_part01_text-generation.ipynb) - - [Part 2: Deep Dream](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter12_part02_deep-dream.ipynb) - - [Part 3: Neural style transfer](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter12_part03_neural-style-transfer.ipynb) - - [Part 4: Variational autoencoders](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter12_part04_variational-autoencoders.ipynb) - - [Part 5: Generative adversarial networks](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter12_part05_gans.ipynb) -* [Chapter 13: Best practices for the real world](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter13_best-practices-for-the-real-world.ipynb) -* [Chapter 14: Conclusions](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter14_conclusions.ipynb) +* [Chapter 7: A deep dive on Keras](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter07_deep-dive-keras.ipynb) +* [Chapter 8: Image Classification](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter08_image-classification.ipynb) +* [Chapter 9: Convnet architecture patterns](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter09_convnet-architecture-patterns.ipynb) +* [Chapter 10: Interpreting what ConvNets learn](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter10_interpreting-what-convnets-learn.ipynb) +* [Chapter 11: Image Segmentation](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter11_image-segmentation.ipynb) +* [Chapter 12: Object Detection](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter12_object-detection.ipynb) +* [Chapter 13: Timeseries Forecasting](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter13_timeseries-forecasting.ipynb) +* [Chapter 14: Text Classification](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter14_text-classification.ipynb) +* [Chapter 15: Language Models and the Transformer](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter15_language-models-and-the-transformer.ipynb) +* [Chapter 16: Text Generation](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter16_text-generation.ipynb) +* [Chapter 17: Image Generation](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter17_image-generation.ipynb) +* [Chapter 18: Best practices for the real world](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/chapter18_best-practices-for-the-real-world.ipynb) diff --git a/chapter02_mathematical-building-blocks.ipynb b/chapter02_mathematical-building-blocks.ipynb index a0ad2070d..de53b0dd6 100644 --- a/chapter02_mathematical-building-blocks.ipynb +++ b/chapter02_mathematical-building-blocks.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -29,7 +29,41 @@ "outputs": [], "source": [ "import os\n", - "os.environ[\"KERAS_BACKEND\"] = \"jax\"" + "os.environ[\"KERAS_BACKEND\"] = \"tensorflow\"" + ] + }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## The mathematical building blocks of neural networks" ] }, { @@ -958,7 +992,7 @@ "colab_type": "text" }, "source": [ - "### The engine of neural networks: gradient-based optimization" + "### The engine of neural networks: Gradient-based optimization" ] }, { @@ -994,7 +1028,7 @@ "colab_type": "text" }, "source": [ - "#### Chaining derivatives: the Backpropagation algorithm" + "#### Chaining derivatives: The Backpropagation algorithm" ] }, { @@ -1104,17 +1138,6 @@ "##### A simple Dense class" ] }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ - "keras.config.set_backend(\"tensorflow\")" - ] - }, { "cell_type": "code", "execution_count": 0, @@ -1299,6 +1322,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "import tensorflow as tf\n", "\n", "x = tf.zeros(shape=())\n", @@ -1315,6 +1339,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "def one_training_step(model, images_batch, labels_batch):\n", " with tf.GradientTape() as tape:\n", " predictions = model(images_batch)\n", @@ -1342,6 +1367,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "def fit(model, images, labels, epochs, batch_size=128):\n", " for epoch_counter in range(epochs):\n", " print(f\"Epoch {epoch_counter}\")\n", @@ -1361,6 +1387,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "from keras.datasets import mnist\n", "\n", "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n", @@ -1390,20 +1417,12 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "predictions = model(test_images)\n", "predicted_labels = ops.argmax(predictions, axis=1)\n", "matches = predicted_labels == test_labels\n", "f\"accuracy: {ops.mean(matches):.2f}\"" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter03_introduction-to-ml-frameworks.ipynb b/chapter03_introduction-to-ml-frameworks.ipynb index 523816aec..7d29c2f85 100644 --- a/chapter03_introduction-to-ml-frameworks.ipynb +++ b/chapter03_introduction-to-ml-frameworks.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Introduction to TensorFlow, PyTorch, JAX, and Keras" + ] + }, { "cell_type": "markdown", "metadata": { @@ -74,7 +108,7 @@ "colab_type": "text" }, "source": [ - "##### Tensors and Variables in TensorFlow" + "##### Tensors and variables in TensorFlow" ] }, { @@ -159,7 +193,7 @@ "colab_type": "text" }, "source": [ - "###### Tensor assignment and the `Variable` class" + "###### Tensor assignment and the Variable class" ] }, { @@ -227,7 +261,7 @@ "colab_type": "text" }, "source": [ - "##### Tensor operations: doing math in TensorFlow" + "##### Tensor operations: Doing math in TensorFlow" ] }, { @@ -264,7 +298,7 @@ "colab_type": "text" }, "source": [ - "##### Gradients in TensorFlow: a second look at the `GradientTape` API" + "##### Gradients in TensorFlow: A second look at the GradientTape API" ] }, { @@ -353,7 +387,7 @@ "colab_type": "text" }, "source": [ - "#### An end-to-end example: a linear classifier in pure TensorFlow" + "#### An end-to-end example: A linear classifier in pure TensorFlow" ] }, { @@ -549,7 +583,7 @@ "colab_type": "text" }, "source": [ - "##### Tensors and Parameters in PyTorch" + "##### Tensors and parameters in PyTorch" ] }, { @@ -668,7 +702,7 @@ "colab_type": "text" }, "source": [ - "##### Tensor operations: doing math in PyTorch" + "##### Tensor operations: Doing math in PyTorch" ] }, { @@ -753,7 +787,7 @@ "colab_type": "text" }, "source": [ - "#### An end-to-end example: a linear classifier in pure PyTorch" + "#### An end-to-end example: A linear classifier in pure PyTorch" ] }, { @@ -825,7 +859,7 @@ "colab_type": "text" }, "source": [ - "##### Packaging state and computation with the `Module` class" + "##### Packaging state and computation with the Module class" ] }, { @@ -1149,7 +1183,7 @@ "colab_type": "text" }, "source": [ - "##### Tensor operations: doing math in JAX" + "##### Tensor operations: Doing math in JAX" ] }, { @@ -1278,7 +1312,7 @@ "colab_type": "text" }, "source": [ - "##### Making JAX functions fast with `@jax.jit`" + "##### Making JAX functions fast with @jax.jit" ] }, { @@ -1300,7 +1334,7 @@ "colab_type": "text" }, "source": [ - "#### An end-to-end example: a linear classifier in pure JAX" + "#### An end-to-end example: A linear classifier in pure JAX" ] }, { @@ -1440,7 +1474,7 @@ "colab_type": "text" }, "source": [ - "#### Layers: the building blocks of deep learning" + "#### Layers: The building blocks of deep learning" ] }, { @@ -1502,7 +1536,7 @@ "colab_type": "text" }, "source": [ - "##### Automatic shape inference: building layers on the fly" + "##### Automatic shape inference: Building layers on the fly" ] }, { @@ -1570,7 +1604,7 @@ "colab_type": "text" }, "source": [ - "#### The \"compile\" step: configuring the learning process" + "#### The \"compile\" step: Configuring the learning process" ] }, { @@ -1619,7 +1653,7 @@ "colab_type": "text" }, "source": [ - "#### Understanding the `fit` method" + "#### Understanding the fit method" ] }, { @@ -1655,7 +1689,7 @@ "colab_type": "text" }, "source": [ - "#### Monitoring loss & metrics on validation data" + "#### Monitoring loss and metrics on validation data" ] }, { @@ -1697,7 +1731,7 @@ "colab_type": "text" }, "source": [ - "#### Inference: using a model after training" + "#### Inference: Using a model after training" ] }, { @@ -1711,15 +1745,6 @@ "predictions = model.predict(val_inputs, batch_size=128)\n", "print(predictions[:10])" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter04_classification-and-regression.ipynb b/chapter04_classification-and-regression.ipynb index b4c0d81ec..1e2e7e822 100644 --- a/chapter04_classification-and-regression.ipynb +++ b/chapter04_classification-and-regression.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,13 +32,47 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Classification and regression" + ] + }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ - "### Classifying movie reviews: a binary classification example" + "### Classifying movie reviews: A binary classification example" ] }, { @@ -413,7 +447,7 @@ "colab_type": "text" }, "source": [ - "### Classifying newswires: a multiclass classification example" + "### Classifying newswires: A multiclass classification example" ] }, { @@ -1237,15 +1271,6 @@ "source": [ "#### Wrapping up" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter05_fundamentals-of-ml.ipynb b/chapter05_fundamentals-of-ml.ipynb index 2e152b214..2aadcc9b8 100644 --- a/chapter05_fundamentals-of-ml.ipynb +++ b/chapter05_fundamentals-of-ml.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,13 +32,47 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Fundamentals of machine learning" + ] + }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ - "### Generalization: the goal of machine learning" + "### Generalization: The goal of machine learning" ] }, { @@ -397,7 +431,7 @@ "colab_type": "text" }, "source": [ - "#### Leveraging better architecture priors" + "#### Using better architecture priors" ] }, { @@ -894,7 +928,7 @@ "outputs": [], "source": [ "original_val_loss = history_original.history[\"val_loss\"]\n", - "l2_val_loss = history_l2_reg.history[\"val_loss\"]\n", + "dropout_val_loss = history_dropout.history[\"val_loss\"]\n", "epochs = range(1, 21)\n", "plt.plot(\n", " epochs,\n", @@ -904,7 +938,7 @@ ")\n", "plt.plot(\n", " epochs,\n", - " l2_val_loss,\n", + " dropout_val_loss,\n", " \"b-\",\n", " label=\"Validation loss of dropout-regularized model\",\n", ")\n", @@ -917,15 +951,6 @@ "plt.legend()\n", "plt.show()" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter07_deep-dive-keras.ipynb b/chapter07_deep-dive-keras.ipynb index 855fd0e61..be5963473 100644 --- a/chapter07_deep-dive-keras.ipynb +++ b/chapter07_deep-dive-keras.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## A deep dive on Keras" + ] + }, { "cell_type": "markdown", "metadata": { @@ -410,7 +444,7 @@ "colab_type": "text" }, "source": [ - "##### The power of the Functional API: access to layer connectivity" + "##### The power of the Functional API: Access to layer connectivity" ] }, { @@ -529,7 +563,7 @@ "colab_type": "text" }, "source": [ - "#### Subclassing the `Model` class" + "#### Subclassing the Model class" ] }, { @@ -619,7 +653,7 @@ "colab_type": "text" }, "source": [ - "##### Beware: what subclassed models don't support" + "##### Beware: What subclassed models don't support" ] }, { @@ -690,7 +724,7 @@ "colab_type": "text" }, "source": [ - "#### Remember: use the right tool for the job" + "#### Remember: Use the right tool for the job" ] }, { @@ -813,7 +847,7 @@ "colab_type": "text" }, "source": [ - "#### Using Callbacks" + "#### Using callbacks" ] }, { @@ -822,7 +856,7 @@ "colab_type": "text" }, "source": [ - "##### The `EarlyStopping` and `ModelCheckpoint` callbacks" + "##### The EarlyStopping and ModelCheckpoint callbacks" ] }, { @@ -996,7 +1030,7 @@ "colab_type": "text" }, "source": [ - "#### Training versus inference" + "#### Training vs. inference" ] }, { @@ -1025,28 +1059,7 @@ }, "outputs": [], "source": [ - "keras.config.set_backend(\"tensorflow\")\n", - "\n", - "import keras\n", - "from keras import layers\n", - "\n", - "def get_mnist_model():\n", - " inputs = keras.Input(shape=(28 * 28,))\n", - " features = layers.Dense(512, activation=\"relu\")(inputs)\n", - " features = layers.Dropout(0.5)(features)\n", - " outputs = layers.Dense(10, activation=\"softmax\")(features)\n", - " model = keras.Model(inputs, outputs)\n", - " return model" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ + "%%backend tensorflow\n", "import tensorflow as tf\n", "\n", "model = get_mnist_model()\n", @@ -1070,6 +1083,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "batch_size = 32\n", "inputs = train_images[:batch_size]\n", "targets = train_labels[:batch_size]\n", @@ -1093,28 +1107,7 @@ }, "outputs": [], "source": [ - "keras.config.set_backend(\"torch\")\n", - "\n", - "import keras\n", - "from keras import layers\n", - "\n", - "def get_mnist_model():\n", - " inputs = keras.Input(shape=(28 * 28,))\n", - " features = layers.Dense(512, activation=\"relu\")(inputs)\n", - " features = layers.Dropout(0.5)(features)\n", - " outputs = layers.Dense(10, activation=\"softmax\")(features)\n", - " model = keras.Model(inputs, outputs)\n", - " return model" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ + "%%backend torch\n", "import torch\n", "\n", "model = get_mnist_model()\n", @@ -1140,6 +1133,7 @@ }, "outputs": [], "source": [ + "%%backend torch\n", "batch_size = 32\n", "inputs = train_images[:batch_size]\n", "targets = train_labels[:batch_size]\n", @@ -1163,28 +1157,7 @@ }, "outputs": [], "source": [ - "keras.config.set_backend(\"jax\")\n", - "\n", - "import keras\n", - "from keras import layers\n", - "\n", - "def get_mnist_model():\n", - " inputs = keras.Input(shape=(28 * 28,))\n", - " features = layers.Dense(512, activation=\"relu\")(inputs)\n", - " features = layers.Dropout(0.5)(features)\n", - " outputs = layers.Dense(10, activation=\"softmax\")(features)\n", - " model = keras.Model(inputs, outputs)\n", - " return model" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ + "%%backend jax\n", "model = get_mnist_model()\n", "loss_fn = keras.losses.SparseCategoricalCrossentropy()\n", "\n", @@ -1206,6 +1179,7 @@ }, "outputs": [], "source": [ + "%%backend jax\n", "import jax\n", "\n", "grad_fn = jax.value_and_grad(compute_loss_and_updates, has_aux=True)" @@ -1219,6 +1193,7 @@ }, "outputs": [], "source": [ + "%%backend jax\n", "optimizer = keras.optimizers.Adam()\n", "optimizer.build(model.trainable_variables)\n", "\n", @@ -1245,6 +1220,7 @@ }, "outputs": [], "source": [ + "%%backend jax\n", "batch_size = 32\n", "inputs = train_images[:batch_size]\n", "targets = train_labels[:batch_size]\n", @@ -1327,7 +1303,7 @@ "colab_type": "text" }, "source": [ - "#### Leveraging `fit()` with a custom training loop" + "#### Using fit() with a custom training loop" ] }, { @@ -1336,18 +1312,7 @@ "colab_type": "text" }, "source": [ - "##### Customizing `fit()` with TensorFlow" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ - "keras.config.set_backend(\"tensorflow\")" + "##### Customizing fit() with TensorFlow" ] }, { @@ -1358,6 +1323,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "import keras\n", "from keras import layers\n", "\n", @@ -1389,6 +1355,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "def get_custom_model():\n", " inputs = keras.Input(shape=(28 * 28,))\n", " features = layers.Dense(512, activation=\"relu\")(inputs)\n", @@ -1407,6 +1374,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "model = get_custom_model()\n", "model.fit(train_images, train_labels, epochs=3)" ] @@ -1417,18 +1385,7 @@ "colab_type": "text" }, "source": [ - "##### Customizing `fit()` with PyTorch" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ - "keras.config.set_backend(\"torch\")" + "##### Customizing fit() with PyTorch" ] }, { @@ -1439,6 +1396,7 @@ }, "outputs": [], "source": [ + "%%backend torch\n", "import keras\n", "from keras import layers\n", "\n", @@ -1474,28 +1432,37 @@ }, "outputs": [], "source": [ - "model = get_custom_model()\n", - "model.fit(train_images, train_labels, epochs=3)" + "%%backend torch\n", + "def get_custom_model():\n", + " inputs = keras.Input(shape=(28 * 28,))\n", + " features = layers.Dense(512, activation=\"relu\")(inputs)\n", + " features = layers.Dropout(0.5)(features)\n", + " outputs = layers.Dense(10, activation=\"softmax\")(features)\n", + " model = CustomModel(inputs, outputs)\n", + " model.compile(optimizer=keras.optimizers.Adam())\n", + " return model" ] }, { - "cell_type": "markdown", + "cell_type": "code", + "execution_count": 0, "metadata": { - "colab_type": "text" + "colab_type": "code" }, + "outputs": [], "source": [ - "##### Customizing `fit()` with JAX" + "%%backend torch\n", + "model = get_custom_model()\n", + "model.fit(train_images, train_labels, epochs=3)" ] }, { - "cell_type": "code", - "execution_count": 0, + "cell_type": "markdown", "metadata": { - "colab_type": "code" + "colab_type": "text" }, - "outputs": [], "source": [ - "keras.config.set_backend(\"jax\")" + "##### Customizing fit() with JAX" ] }, { @@ -1506,6 +1473,7 @@ }, "outputs": [], "source": [ + "%%backend jax\n", "import keras\n", "from keras import layers\n", "\n", @@ -1575,17 +1543,28 @@ }, "outputs": [], "source": [ - "model = get_custom_model()\n", - "model.fit(train_images, train_labels, epochs=3)" + "%%backend jax\n", + "def get_custom_model():\n", + " inputs = keras.Input(shape=(28 * 28,))\n", + " features = layers.Dense(512, activation=\"relu\")(inputs)\n", + " features = layers.Dropout(0.5)(features)\n", + " outputs = layers.Dense(10, activation=\"softmax\")(features)\n", + " model = CustomModel(inputs, outputs)\n", + " model.compile(optimizer=keras.optimizers.Adam())\n", + " return model" ] }, { - "cell_type": "markdown", + "cell_type": "code", + "execution_count": 0, "metadata": { - "colab_type": "text" + "colab_type": "code" }, + "outputs": [], "source": [ - "#### Handling metrics in a custom `train_step()`" + "%%backend jax\n", + "model = get_custom_model()\n", + "model.fit(train_images, train_labels, epochs=3)" ] }, { @@ -1594,18 +1573,16 @@ "colab_type": "text" }, "source": [ - "##### `train_step()` metrics handling with TensorFlow" + "#### Handling metrics in a custom train_step()" ] }, { - "cell_type": "code", - "execution_count": 0, + "cell_type": "markdown", "metadata": { - "colab_type": "code" + "colab_type": "text" }, - "outputs": [], "source": [ - "keras.config.set_backend(\"tensorflow\")" + "##### train_step() metrics handling with TensorFlow" ] }, { @@ -1616,6 +1593,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "import keras\n", "from keras import layers\n", "\n", @@ -1646,6 +1624,7 @@ }, "outputs": [], "source": [ + "%%backend tensorflow\n", "def get_custom_model():\n", " inputs = keras.Input(shape=(28 * 28,))\n", " features = layers.Dense(512, activation=\"relu\")(inputs)\n", @@ -1669,18 +1648,7 @@ "colab_type": "text" }, "source": [ - "##### `train_step()` metrics handling with PyTorch" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ - "keras.config.set_backend(\"torch\")" + "##### train_step() metrics handling with PyTorch" ] }, { @@ -1691,6 +1659,7 @@ }, "outputs": [], "source": [ + "%%backend torch\n", "import keras\n", "from keras import layers\n", "\n", @@ -1724,6 +1693,7 @@ }, "outputs": [], "source": [ + "%%backend torch\n", "def get_custom_model():\n", " inputs = keras.Input(shape=(28 * 28,))\n", " features = layers.Dense(512, activation=\"relu\")(inputs)\n", @@ -1747,18 +1717,7 @@ "colab_type": "text" }, "source": [ - "##### `train_step()` metrics handling with JAX" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ - "keras.config.set_backend(\"jax\")" + "##### train_step() metrics handling with JAX" ] }, { @@ -1769,6 +1728,7 @@ }, "outputs": [], "source": [ + "%%backend jax\n", "import keras\n", "from keras import layers\n", "\n", @@ -1840,15 +1800,6 @@ " )\n", " return logs, state" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter08_image-classification.ipynb b/chapter08_image-classification.ipynb index 40320aeaf..3ce065e2b 100644 --- a/chapter08_image-classification.ipynb +++ b/chapter08_image-classification.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Image classification" + ] + }, { "cell_type": "markdown", "metadata": { @@ -644,7 +678,7 @@ "colab_type": "text" }, "source": [ - "### Leveraging a pretrained model" + "### Using a pretrained model" ] }, { @@ -962,15 +996,6 @@ "test_loss, test_acc = model.evaluate(test_dataset)\n", "print(f\"Test accuracy: {test_acc:.3f}\")" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter09_convnet-architecture-patterns.ipynb b/chapter09_convnet-architecture-patterns.ipynb index 60c3a9ce0..218546a3c 100644 --- a/chapter09_convnet-architecture-patterns.ipynb +++ b/chapter09_convnet-architecture-patterns.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Convnet architecture patterns" + ] + }, { "cell_type": "markdown", "metadata": { @@ -142,7 +176,7 @@ "colab_type": "text" }, "source": [ - "### Putting it together: a mini Xception-like model" + "### Putting it together: A mini Xception-like model" ] }, { @@ -313,15 +347,6 @@ "source": [ "### Beyond convolution: Vision Transformers" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter10_interpreting-what-convnets-learn.ipynb b/chapter10_interpreting-what-convnets-learn.ipynb index 64d98c05a..1ff016432 100644 --- a/chapter10_interpreting-what-convnets-learn.ipynb +++ b/chapter10_interpreting-what-convnets-learn.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Interpreting what convnets learn" + ] + }, { "cell_type": "markdown", "metadata": { @@ -304,32 +338,7 @@ }, "outputs": [], "source": [ - "keras.config.set_backend(\"tensorflow\")\n", - "\n", - "import keras\n", - "import keras_hub\n", - "from keras import ops\n", - "\n", - "model = keras_hub.models.Backbone.from_preset(\"xception_41_imagenet\")\n", - "\n", - "layer_name = \"block3_sepconv1\"\n", - "layer = model.get_layer(name=layer_name)\n", - "feature_extractor = keras.Model(inputs=model.input, outputs=layer.output)\n", - "\n", - "def compute_loss(image, filter_index):\n", - " activation = feature_extractor(image)\n", - " filter_activation = activation[:, 2:-2, 2:-2, filter_index]\n", - " return ops.mean(filter_activation)" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ + "%%backend tensorflow\n", "import tensorflow as tf\n", "\n", "@tf.function\n", @@ -360,32 +369,7 @@ }, "outputs": [], "source": [ - "keras.config.set_backend(\"torch\")\n", - "\n", - "import keras\n", - "import keras_hub\n", - "from keras import ops\n", - "\n", - "model = keras_hub.models.Backbone.from_preset(\"xception_41_imagenet\")\n", - "\n", - "layer_name = \"block3_sepconv1\"\n", - "layer = model.get_layer(name=layer_name)\n", - "feature_extractor = keras.Model(inputs=model.input, outputs=layer.output)\n", - "\n", - "def compute_loss(image, filter_index):\n", - " activation = feature_extractor(image)\n", - " filter_activation = activation[:, 2:-2, 2:-2, filter_index]\n", - " return ops.mean(filter_activation)" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ + "%%backend torch\n", "import torch\n", "\n", "def gradient_ascent_step(image, filter_index, learning_rate):\n", @@ -415,32 +399,7 @@ }, "outputs": [], "source": [ - "keras.config.set_backend(\"jax\")\n", - "\n", - "import keras\n", - "import keras_hub\n", - "from keras import ops\n", - "\n", - "model = keras_hub.models.Backbone.from_preset(\"xception_41_imagenet\")\n", - "\n", - "layer_name = \"block3_sepconv1\"\n", - "layer = model.get_layer(name=layer_name)\n", - "feature_extractor = keras.Model(inputs=model.input, outputs=layer.output)\n", - "\n", - "def compute_loss(image, filter_index):\n", - " activation = feature_extractor(image)\n", - " filter_activation = activation[:, 2:-2, 2:-2, filter_index]\n", - " return ops.mean(filter_activation)" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ + "%%backend jax\n", "import jax\n", "\n", "grad_fn = jax.grad(compute_loss)\n", @@ -668,36 +627,7 @@ }, "outputs": [], "source": [ - "keras.config.set_backend(\"tensorflow\")\n", - "\n", - "import keras\n", - "import keras_hub\n", - "from keras import ops\n", - "\n", - "model = keras_hub.models.ImageClassifier.from_preset(\n", - " \"xception_41_imagenet\",\n", - " activation=\"softmax\",\n", - ")\n", - "\n", - "last_conv_layer_name = \"block14_sepconv2_act\"\n", - "last_conv_layer = model.backbone.get_layer(last_conv_layer_name)\n", - "last_conv_layer_model = keras.Model(model.inputs, last_conv_layer.output)\n", - "\n", - "classifier_input = last_conv_layer.output\n", - "x = classifier_input\n", - "for layer_name in [\"pooler\", \"predictions\"]:\n", - " x = model.get_layer(layer_name)(x)\n", - "classifier_model = keras.Model(classifier_input, x)" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ + "%%backend tensorflow\n", "import tensorflow as tf\n", "\n", "def get_top_class_gradients(img_array):\n", @@ -733,36 +663,7 @@ }, "outputs": [], "source": [ - "keras.config.set_backend(\"torch\")\n", - "\n", - "import keras\n", - "import keras_hub\n", - "from keras import ops\n", - "\n", - "model = keras_hub.models.ImageClassifier.from_preset(\n", - " \"xception_41_imagenet\",\n", - " activation=\"softmax\",\n", - ")\n", - "\n", - "last_conv_layer_name = \"block14_sepconv2_act\"\n", - "last_conv_layer = model.backbone.get_layer(last_conv_layer_name)\n", - "last_conv_layer_model = keras.Model(model.inputs, last_conv_layer.output)\n", - "\n", - "classifier_input = last_conv_layer.output\n", - "x = classifier_input\n", - "for layer_name in [\"pooler\", \"predictions\"]:\n", - " x = model.get_layer(layer_name)(x)\n", - "classifier_model = keras.Model(classifier_input, x)" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ + "%%backend torch\n", "def get_top_class_gradients(img_array):\n", " last_conv_layer_output = last_conv_layer_model(img_array)\n", " last_conv_layer_output = (\n", @@ -797,36 +698,7 @@ }, "outputs": [], "source": [ - "keras.config.set_backend(\"jax\")\n", - "\n", - "import keras\n", - "import keras_hub\n", - "from keras import ops\n", - "\n", - "model = keras_hub.models.ImageClassifier.from_preset(\n", - " \"xception_41_imagenet\",\n", - " activation=\"softmax\",\n", - ")\n", - "\n", - "last_conv_layer_name = \"block14_sepconv2_act\"\n", - "last_conv_layer = model.backbone.get_layer(last_conv_layer_name)\n", - "last_conv_layer_model = keras.Model(model.inputs, last_conv_layer.output)\n", - "\n", - "classifier_input = last_conv_layer.output\n", - "x = classifier_input\n", - "for layer_name in [\"pooler\", \"predictions\"]:\n", - " x = model.get_layer(layer_name)(x)\n", - "classifier_model = keras.Model(classifier_input, x)" - ] - }, - { - "cell_type": "code", - "execution_count": 0, - "metadata": { - "colab_type": "code" - }, - "outputs": [], - "source": [ + "%%backend jax\n", "import jax\n", "\n", "def loss_fn(last_conv_layer_output):\n", @@ -910,8 +782,7 @@ "superimposed_img = jet_heatmap * 0.4 + img\n", "superimposed_img = keras.utils.array_to_img(superimposed_img)\n", "\n", - "save_path = \"elephant_cam.jpg\"\n", - "superimposed_img.save(save_path)" + "plt.imshow(superimposed_img)" ] }, { @@ -922,15 +793,6 @@ "source": [ "### Visualizing the latent space of a convnet" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter11_image-segmentation.ipynb b/chapter11_image-segmentation.ipynb index 607ab9b02..74f4c6b13 100644 --- a/chapter11_image-segmentation.ipynb +++ b/chapter11_image-segmentation.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Image segmentation" + ] + }, { "cell_type": "markdown", "metadata": { @@ -225,6 +259,19 @@ "model = get_model(img_size=img_size, num_classes=3)" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# \u26a0\ufe0fNOTE\u26a0\ufe0f: The following IoU metric is *very* slow on the PyTorch backend!\n", + "# If you are running with PyTorch, we recommend re-running the notebook with Jax\n", + "# or TensorFlow, or skipping to the next section of this chapter." + ] + }, { "cell_type": "code", "execution_count": 0, @@ -330,7 +377,7 @@ "colab_type": "text" }, "source": [ - "#### Downloading the Segment Anything model" + "#### Downloading the Segment Anything Model" ] }, { @@ -372,7 +419,7 @@ "colab_type": "text" }, "source": [ - "#### Prepare a test image" + "#### Preparing a test image" ] }, { @@ -620,15 +667,6 @@ "show_box(input_box, plt.gca())\n", "plt.show()" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter12_object-detection.ipynb b/chapter12_object-detection.ipynb index 1ce547994..6b562082e 100644 --- a/chapter12_object-detection.ipynb +++ b/chapter12_object-detection.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,13 +32,47 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Object detection" + ] + }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ - "### Single-stage vs two-stage object detectors" + "### Single-stage vs. two-stage object detectors" ] }, { @@ -648,15 +682,6 @@ " draw_box(ax, box, label_name, label_to_color(label))\n", "plt.show()" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter13_timeseries-forecasting.ipynb b/chapter13_timeseries-forecasting.ipynb index 0413202fc..0fe4788f1 100644 --- a/chapter13_timeseries-forecasting.ipynb +++ b/chapter13_timeseries-forecasting.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Timeseries forecasting" + ] + }, { "cell_type": "markdown", "metadata": { @@ -646,15 +680,6 @@ "source": [ "### Going even further" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter14_text-classification.ipynb b/chapter14_text-classification.ipynb index 7f0e90c7b..15e34f0f0 100644 --- a/chapter14_text-classification.ipynb +++ b/chapter14_text-classification.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,13 +32,47 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Text classification" + ] + }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ - "### A brief history of Natural Language Processing" + "### A brief history of natural language processing" ] }, { @@ -586,7 +620,7 @@ "colab_type": "text" }, "source": [ - "### Sets vs. Sequences" + "### Sets vs. sequences" ] }, { @@ -595,7 +629,7 @@ "colab_type": "text" }, "source": [ - "#### Loading the IMDb Classification Dataset" + "#### Loading the IMDb classification dataset" ] }, { @@ -1072,6 +1106,22 @@ "model.summary(line_length=80)" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# \u26a0\ufe0fNOTE\u26a0\ufe0f: The following fit call will error on a T4 GPU on the TensorFlow\n", + "# backend due to a bug in TensorFlow. If you the follow cell errors out,\n", + "# do one of the following:\n", + "# - Skip the following two cells.\n", + "# - Switch to the Jax or Torch backend and re-run this notebook.\n", + "# - Change the GPU type in your runtime (requires Colab Pro as of this writing)." + ] + }, { "cell_type": "code", "execution_count": 0, @@ -1355,15 +1405,6 @@ "test_loss, test_acc = model.evaluate(sequence_test_ds)\n", "test_acc" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter15_language-models-and-the-transformer.ipynb b/chapter15_language-models-and-the-transformer.ipynb index dc2a06520..d43b72e86 100644 --- a/chapter15_language-models-and-the-transformer.ipynb +++ b/chapter15_language-models-and-the-transformer.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,13 +32,47 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Language models and the Transformer" + ] + }, { "cell_type": "markdown", "metadata": { "colab_type": "text" }, "source": [ - "### The Language Model" + "### The language model" ] }, { @@ -47,7 +81,7 @@ "colab_type": "text" }, "source": [ - "#### Training a Shakespeare Language Model" + "#### Training a Shakespeare language model" ] }, { @@ -315,7 +349,7 @@ "colab_type": "text" }, "source": [ - "#### English to Spanish Translation" + "#### English-to-Spanish Translation" ] }, { @@ -632,7 +666,7 @@ "colab_type": "text" }, "source": [ - "#### Transformer Encoder block" + "#### Transformer encoder block" ] }, { @@ -673,7 +707,7 @@ "colab_type": "text" }, "source": [ - "#### Transformer Decoder block" + "#### Transformer decoder block" ] }, { @@ -967,7 +1001,7 @@ "colab_type": "text" }, "source": [ - "#### Preprocessing IMDb Movie reviews" + "#### Preprocessing IMDb movie reviews" ] }, { @@ -1125,15 +1159,6 @@ "source": [ "### What makes the Transformer effective?" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter16_text-generation.ipynb b/chapter16_text-generation.ipynb index 8f6f0c94c..06ce65647 100644 --- a/chapter16_text-generation.ipynb +++ b/chapter16_text-generation.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io).\n" ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Text generation" + ] + }, { "cell_type": "markdown", "metadata": { @@ -938,9 +972,9 @@ }, "outputs": [], "source": [ - "# NOTE: If you are running on the free tier Colab GPUs, you will need to restart\n", - "# your runtime and run the notebook from here to free up memory for this\n", - "# 4 billion parameter model.\n", + "# \u26a0\ufe0fNOTE\u26a0\ufe0f: If you are running on the free tier Colab GPUs, you will need to\n", + "# restart your runtime and run the notebook from here to free up memory for\n", + "# this 4 billion parameter model.\n", "import os\n", "\n", "os.environ[\"KERAS_BACKEND\"] = \"jax\"\n", @@ -1142,15 +1176,6 @@ "source": [ "### Where are LLMs heading next?" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter17_image-generation.ipynb b/chapter17_image-generation.ipynb index d69809894..f78b56050 100644 --- a/chapter17_image-generation.ipynb +++ b/chapter17_image-generation.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Image generation" + ] + }, { "cell_type": "markdown", "metadata": { @@ -400,7 +434,7 @@ "colab_type": "text" }, "source": [ - "#### The concept of \"diffusion time\" and \"diffusion schedule\"" + "#### The concepts of diffusion time and diffusion schedule" ] }, { @@ -834,15 +868,6 @@ " images.append(scale_output(image))\n", "display(images)" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/chapter18_best-practices-for-the-real-world.ipynb b/chapter18_best-practices-for-the-real-world.ipynb index 33a0adbda..d7e28359a 100644 --- a/chapter18_best-practices-for-the-real-world.ipynb +++ b/chapter18_best-practices-for-the-real-world.ipynb @@ -6,7 +6,7 @@ "colab_type": "text" }, "source": [ - "This is a companion notebook for the book [Deep Learning with Python, Third Edition](TODO). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**" + "This is a companion notebook for the book [Deep Learning with Python, Third Edition](https://www.manning.com/books/deep-learning-with-python-third-edition). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.\n\n**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**\n\nThe book's contents are available online at [deeplearningwithpython.io](https://deeplearningwithpython.io)." ] }, { @@ -32,6 +32,40 @@ "os.environ[\"KERAS_BACKEND\"] = \"jax\"" ] }, + { + "cell_type": "code", + "execution_count": 0, + "metadata": { + "cellView": "form", + "colab_type": "code" + }, + "outputs": [], + "source": [ + "# @title\n", + "import os\n", + "from IPython.core.magic import register_cell_magic\n", + "\n", + "@register_cell_magic\n", + "def backend(line, cell):\n", + " current, required = os.environ.get(\"KERAS_BACKEND\", \"\"), line.split()[-1]\n", + " if current == required:\n", + " get_ipython().run_cell(cell)\n", + " else:\n", + " print(\n", + " f\"This cell requires the {required} backend. To run it, change KERAS_BACKEND to \"\n", + " f\"\\\"{required}\\\" at the top of the notebook, restart the runtime, and rerun the notebook.\"\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "## Best practices for the real world" + ] + }, { "cell_type": "markdown", "metadata": { @@ -316,7 +350,25 @@ "colab_type": "text" }, "source": [ - "##### Data parallelism: replicate your model on each GPU" + "##### Data parallelism: Replicating your model on each GPU" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "##### Model parallelism: Splitting your model across multiple GPUs" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text" + }, + "source": [ + "#### Distributed training in practice" ] }, { @@ -325,7 +377,7 @@ "colab_type": "text" }, "source": [ - "##### Model parallelism: split your model across multiple GPUs" + "##### Getting your hands on two or more GPUs" ] }, { @@ -334,7 +386,7 @@ "colab_type": "text" }, "source": [ - "##### Distributed training in practice" + "##### Using data parallelism with JAX" ] }, { @@ -343,7 +395,7 @@ "colab_type": "text" }, "source": [ - "###### Getting your hands on two or more GPUs" + "##### Using model parallelism with JAX" ] }, { @@ -352,7 +404,7 @@ "colab_type": "text" }, "source": [ - "###### Using data parallelism with JAX" + "###### The DeviceMesh API" ] }, { @@ -361,7 +413,7 @@ "colab_type": "text" }, "source": [ - "###### Using model parallelism with JAX" + "###### The LayoutMap API" ] }, { @@ -379,7 +431,7 @@ "colab_type": "text" }, "source": [ - "##### Leveraging step fusing to improve TPU utilization" + "##### Using step fusing to improve TPU utilization" ] }, { @@ -512,15 +564,6 @@ "source": [ "ops.matmul(x, kernel)" ] - }, - { - "cell_type": "markdown", - "metadata": { - "colab_type": "text" - }, - "source": [ - "### Chapter summary" - ] } ], "metadata": { diff --git a/second_edition/README.md b/second_edition/README.md new file mode 100644 index 000000000..53b72c363 --- /dev/null +++ b/second_edition/README.md @@ -0,0 +1,30 @@ +# Second edition notebooks + +These are the notebooks for the second edition of the book, originally published in 2021. These notebooks use `tf.keras` with TensorFlow 2.16. + +## Table of contents + +* [Chapter 2: The mathematical building blocks of neural networks](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter02_mathematical-building-blocks.ipynb) +* [Chapter 3: Introduction to Keras and TensorFlow](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter03_introduction-to-keras-and-tf.ipynb) +* [Chapter 4: Getting started with neural networks: classification and regression](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter04_getting-started-with-neural-networks.ipynb) +* [Chapter 5: Fundamentals of machine learning](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter05_fundamentals-of-ml.ipynb) +* [Chapter 7: Working with Keras: a deep dive](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter07_working-with-keras.ipynb) +* [Chapter 8: Introduction to deep learning for computer vision](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter08_intro-to-dl-for-computer-vision.ipynb) +* Chapter 9: Advanced deep learning for computer vision + - [Part 1: Image segmentation](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter09_part01_image-segmentation.ipynb) + - [Part 2: Modern convnet architecture patterns](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter09_part02_modern-convnet-architecture-patterns.ipynb) + - [Part 3: Interpreting what convnets learn](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter09_part03_interpreting-what-convnets-learn.ipynb) +* [Chapter 10: Deep learning for timeseries](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter10_dl-for-timeseries.ipynb) +* Chapter 11: Deep learning for text + - [Part 1: Introduction](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter11_part01_introduction.ipynb) + - [Part 2: Sequence models](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter11_part02_sequence-models.ipynb) + - [Part 3: Transformer](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter11_part03_transformer.ipynb) + - [Part 4: Sequence-to-sequence learning](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter11_part04_sequence-to-sequence-learning.ipynb) +* Chapter 12: Generative deep learning + - [Part 1: Text generation](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter12_part01_text-generation.ipynb) + - [Part 2: Deep Dream](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter12_part02_deep-dream.ipynb) + - [Part 3: Neural style transfer](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter12_part03_neural-style-transfer.ipynb) + - [Part 4: Variational autoencoders](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter12_part04_variational-autoencoders.ipynb) + - [Part 5: Generative adversarial networks](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter12_part05_gans.ipynb) +* [Chapter 13: Best practices for the real world](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter13_best-practices-for-the-real-world.ipynb) +* [Chapter 14: Conclusions](https://colab.research.google.com/github/fchollet/deep-learning-with-python-notebooks/blob/master/second_edition/chapter14_conclusions.ipynb)