Python - Thread Pools

What is a Thread Pool?

A thread pool is a mechanism that automatically manages a pool of worker threads. Each thread in the pool is called a worker or a worker thread. Worker threads can be re-used once the task is completed. A single thread is able to execute a single task once.

A thread pool controls when the threads are created, and what threads should do when they are not being used.

The pool is significantly efficient to use a thread pool instead of manually starting, managing, and closing threads, especially with a large number of tasks.

Multiple Threads in Python concurrently execute a certain function. Asynchronous execution of a function by multiple threads can be achieved by ThreadPoolExecutor class defined in concurrent.futures module.

The concurrent.futures module includes Future class and two Executor classes − ThreadPoolExecutor and ProcessPoolExecutor.

The Future Class

The concurrent.futures.Future class is responsible for handling asynchronous execution of any callable such as a function. To obtain an object of Future class, you should call the submit() method on any Executor object. It should not be created directly by its constructor.

Important methods in the Future class are −

result(timeout=None)

This method returns the value returned by the call. If the call hasn't yet completed, then this method will wait up to timeout seconds. If the call hasn't completed in timeout seconds, then a TimeoutError will be raised. If timeout is not specified, there is no limit to the wait time.

cancel()

This method makes attempt to cancel the call. If the call is currently being executed or finished running and cannot be cancelled then the method will return False, otherwise the call will be cancelled and the method will return True.

cancelled()

This method returns True if the call was successfully cancelled.

running()

This method returns True if the call is currently being executed and cannot be cancelled.

done()

This method returns True if the call was successfully cancelled or finished running.

The ThreadPoolExecutor Class

This class represents a pool of specified number maximum worker threads to execute calls asynchronously.

concurrent.futures.ThreadPoolExecutor(max_threads)

Example

from concurrent.futures import ThreadPoolExecutor
from time import sleep
def square(numbers):
   for val in numbers:
      ret = val*val
      sleep(1)
      print("Number:{} Square:{}".format(val, ret))
def cube(numbers):
   for val in numbers:
      ret = val*val*val
      sleep(1)
      print("Number:{} Cube:{}".format(val, ret))
if __name__ == '__main__':
   numbers = [1,2,3,4,5]
   executor = ThreadPoolExecutor(4)
   thread1 = executor.submit(square, (numbers))
   thread2 = executor.submit(cube, (numbers))
   print("Thread 1 executed ? :",thread1.done())
   print("Thread 2 executed ? :",thread2.done())
   sleep(2)
   print("Thread 1 executed ? :",thread1.done())
   print("Thread 2 executed ? :",thread2.done())

It will produce the following output −

Thread 1 executed ? : False
Thread 2 executed ? : False
Number:1 Square:1
Number:1 Cube:1
Number:2 Square:4
Number:2 Cube:8
Thread 1 executed ? : False
Thread 2 executed ? : False
Number:3 Square:9
Number:3 Cube:27
Number:4 Square:16
Number:4 Cube:64
Number:5 Square:25
Number:5 Cube:125
Thread 1 executed ? : True
Thread 2 executed ? : True
Advertisements