Multiprocessing in Python

The multiprocessing package supports spawning processes. It refers to a function that loads and executes a new child processes. For the child to terminate or to continue executing concurrent computing,then the current process hasto wait using an API, which is similar to threading module.

Introduction

When we work with Multiprocessing,at first we create process object. Then it calls a start() method.

Example code

from multiprocessing import Process
   def display():
      print ('Hi !! I am Python')
      if __name__ == '__main__':
      p = Process(target=display)
      p.start()
      p.join()

In this example, at first we import the Process class then initiate Process object with the display() function.

Then process is started with start() method and then complete the process with the join() method.

We can also pass arguments to the function using args keyword.

Example

from multiprocessing import Process
   def display(my_name):
   print ('Hi !!!' + " " + my_name)
   if __name__ == '__main__':
      p = Process(target=display, args=('Python',))
      p.start()
      p.join()

In this example, we create a process that calculates the cube of numbers and prints all results to the console.

Example code

from multiprocessing import Process
   def cube(x):
      for x in my_numbers:
         print('%s cube is %s' % (x, x**3))
      if __name__ == '__main__':
         my_numbers = [3, 4, 5, 6, 7, 8]
         p = Process(target=cube, args=('x',))
         p.start()
p.join
print ("Done")

Output

Done
3 cube is 27
4 cube is 64
5 cube is 125
6 cube is 216
7 cube is 343
8 cube is 512

We can also create more than one process at atime.

In this example, at first we create one process which is process1, this process just calculates the cube of a number and at the same time second process process2 is checking that this number is even or odd.

Example

from multiprocessing import Process
def cube(x):
   for x in my_numbers:
   print('%s cube is %s' % (x, x**3))
def evenno(x):
   for x in my_numbers:
   if x % 2 == 0:
   print('%s is an even number ' % (x))
   if __name__ == '__main__':
      my_numbers = [3, 4, 5, 6, 7, 8]
      my_process1 = Process(target=cube, args=('x',))
      my_process2 = Process(target=evenno, args=('x',))
      my_process1.start()
      my_process2.start()
      my_process1.join()
   my_process2.join()
print ("Done")

Output

3 cube is 27
4 cube is 64
5 cube is 125
6 cube is 216
7 cube is 343
8 cube is 512
4 is an even number
6 is an even number
8 is an even number
Done

Communication between Processes

Multiprocessing supports Pipes and Queues, which are two types of communication channels between processes.

Pipes

In multiprocessing, when we want to communicate between processes, in that situation Pipes areused.

Example

from multiprocessing import Process, Pipe
   def myfunction(conn):
      conn.send(['hi!! I am Python'])
      conn.close()
      if __name__ == '__main__':
         parent_conn, child_conn = Pipe()
         p = Process(target=myfunction, args=(child_conn,))
         p.start()
      print (parent_conn.recv() )
p.join()

Output

['hi !!! I am Python']

Pipes return two connection objects and these are representing the two ends of the pipe. Each connection object has two methods one is send() and another one is recv() method.

In this example, at first we create a process and this process prints the message "hi!! I am Python" and then shares the data across.

Queues

When we pass data between processes then at that time we can use Queue object.

Example

import multiprocessing
   def evenno(numbers, q):
      for n in numbers:
      if n % 2 == 0:
      q.put(n)
      if __name__ == "__main__":
         q = multiprocessing.Queue()
         p = multiprocessing.Process(target=evenno, args=(range(10), q))
         p.start()
         p.join()
   while q:
print(q.get())

Output

0
2
4
6
8

In this example, at first create a function that checks weather a number is even or not. If the number is even, then insert it at the end of the queue. Then we create a queue object and a process object then we start the process.

And finally check whether the queue is empty or not.

When we print the numbers, at first we print the value which is in front of the queue then next one and so on.

Locks

When we want that only one process is executed at a time in that situation Locks is use. That means that time blocks other process from executing similar code. Lock will be released after the process gets completed.

Example using Locks method

Example

from multiprocessing import Process, Lock
def dispmay_name(l, i):
l.acquire()
print ('Hi', i)
   l.release()
if __name__ == '__main__':
   my_lock = Lock()
   my_name = ['Aadrika', 'Adwaita', 'Sakya', 'Sanj']
for name in my_name:
Process(target=dispmay_name, args=(my_lock,name)).start()

Output

Hi Aadrika
Hi Adwaita
Hi Sakya
Hi Sanj

Logging

The multiprocessing module also provides logging module to ensure that, if the logging package doesn't use locks function, the messages between processes mixed up during execution.

Example

import multiprocessing, logging
logger = multiprocessing.log_to_stderr()
logger.setLevel(logging.INFO)
logger.warning('Error has occurred')

In this example at first we import the logging and multiprocessing module then we use multiprocessing.log_to_stderr() method. And it call get_logger() as well as adding to sys.stderr and finally we set the level of logger and convey the message.