{
  "__type": "IngestedDoc",
  "__tag": 4010,
  "_content": {},
  "_ordered_sections": [],
  "item_file": null,
  "item_line": null,
  "item_type": null,
  "aliases": [],
  "example_section_data": {
    "__type": "Section",
    "__tag": 4015,
    "children": [],
    "title": [],
    "level": 0,
    "target": null
  },
  "see_also": [],
  "signature": null,
  "references": null,
  "qa": "user_guide:tutorial_parallelization",
  "arbitrary": [
    {
      "__type": "Section",
      "__tag": 4015,
      "children": [
        {
          "__type": "Paragraph",
          "__tag": 4045,
          "children": [
            {
              "__type": "Text",
              "__tag": 4046,
              "value": "In image processing, we frequently apply the same algorithm on a large batch of images. In this paragraph, we propose to use "
            },
            {
              "__type": "Link",
              "__tag": 4049,
              "children": [
                {
                  "__type": "Text",
                  "__tag": 4046,
                  "value": "joblib"
                }
              ],
              "url": "https://joblib.readthedocs.io",
              "title": ""
            },
            {
              "__type": "Text",
              "__tag": 4046,
              "value": " to parallelize loops. Here is an example of such repetitive tasks:"
            }
          ]
        },
        {
          "__type": "Code",
          "__tag": 4050,
          "value": "import skimage as ski\n\ndef task(image):\n    \"\"\"\n    Apply some functions and return an image.\n    \"\"\"\n    image = ski.restoration.denoise_tv_chambolle(\n        image[0][0], weight=0.1, channel_axis=-1\n    )\n    fd, hog_image = ski.feature.hog(\n        ski.color.rgb2gray(image),\n        orientations=8,\n        pixels_per_cell=(16, 16),\n        cells_per_block=(1, 1),\n        visualize=True\n    )\n    return hog_image\n\n\n# Prepare images\nhubble = ski.data.hubble_deep_field()\nwidth = 10\npics = ski.util.view_as_windows(\n    hubble, (width, hubble.shape[1], hubble.shape[2]), step=width\n)",
          "execution_status": null
        },
        {
          "__type": "Paragraph",
          "__tag": 4045,
          "children": [
            {
              "__type": "Text",
              "__tag": 4046,
              "value": "To call the function "
            },
            {
              "__type": "InlineCode",
              "__tag": 4051,
              "value": "task"
            },
            {
              "__type": "Text",
              "__tag": 4046,
              "value": " on each element of the list "
            },
            {
              "__type": "InlineCode",
              "__tag": 4051,
              "value": "pics"
            },
            {
              "__type": "Text",
              "__tag": 4046,
              "value": ", it is usual to write a for loop. To measure the execution time of this loop, you can use ipython and measure the execution time with "
            },
            {
              "__type": "InlineCode",
              "__tag": 4051,
              "value": "%timeit"
            },
            {
              "__type": "Text",
              "__tag": 4046,
              "value": "."
            }
          ]
        },
        {
          "__type": "Code",
          "__tag": 4050,
          "value": "def classic_loop():\n    for image in pics:\n        task(image)\n\n\n%timeit classic_loop()",
          "execution_status": null
        },
        {
          "__type": "Paragraph",
          "__tag": 4045,
          "children": [
            {
              "__type": "Text",
              "__tag": 4046,
              "value": "Another equivalent way to code this loop is to use a comprehension list which has the same efficiency."
            }
          ]
        },
        {
          "__type": "Code",
          "__tag": 4050,
          "value": "def comprehension_loop():\n    [task(image) for image in pics]\n\n%timeit comprehension_loop()",
          "execution_status": null
        },
        {
          "__type": "Paragraph",
          "__tag": 4045,
          "children": [
            {
              "__type": "InlineCode",
              "__tag": 4051,
              "value": "joblib"
            },
            {
              "__type": "Text",
              "__tag": 4046,
              "value": " is a library providing an easy way to parallelize for loops once we have a comprehension list. The number of jobs can be specified."
            }
          ]
        },
        {
          "__type": "Code",
          "__tag": 4050,
          "value": "from joblib import Parallel, delayed\ndef joblib_loop():\n    Parallel(n_jobs=4)(delayed(task)(i) for i in pics)\n\n%timeit joblib_loop()",
          "execution_status": null
        }
      ],
      "title": [
        {
          "__type": "Text",
          "__tag": 4046,
          "value": "How to parallelize loops"
        }
      ],
      "level": 0,
      "target": null
    }
  ],
  "local_refs": []
}