run_tasks()

audeer.run_tasks(task_func, params, *, num_workers=1, multiprocessing=False, progress_bar=False, task_description=None, maximum_refresh_time=None)[source]

Run parallel tasks using multprocessing.

Note

Result values are returned in order of params.

Parameters

  • task_func (Callable) – task function with one or more parameters, e.g. x, y, z, and optionally returning a value

  • params (Sequence[Tuple[Sequence[Any], Dict[str, Any]]]) – sequence of tuples holding parameters for each task. Each tuple contains a sequence of positional arguments and a dictionary with keyword arguments, e.g.: [((x1, y1), {'z': z1}), ((x2, y2), {'z': z2}), ...]

  • num_workers (int) – number of parallel jobs or 1 for sequential processing. If None will be set to the number of processors on the machine multiplied by 5 in case of multithreading and number of processors in case of multiprocessing

  • multiprocessing (bool) – use multiprocessing instead of multithreading

  • progress_bar (bool) – show a progress bar

  • task_description (Optional[str]) – task description that will be displayed next to progress bar

  • maximum_refresh_time (Optional[float]) – refresh the progress bar at least every maximum_refresh_time seconds, using another thread. If None, no refreshing is enforced

Return type

List[Any]

Returns

list of computed results

Examples

>>> power = lambda x, n: x**n
>>> params = [([2, n], {}) for n in range(10)]
>>> run_tasks(power, params, num_workers=3)
[1, 2, 4, 8, 16, 32, 64, 128, 256, 512]