Computing Pi using AsyncResult

Using a process pool makes our example computation of Pi simpler in that we don’t have to create individual processes, but also because we don’t need to use locks to control access and shared data structures to return the values. All of the communication and timing between the processes is taken care of by the AsyncResult objects:

import multiprocessing
import multiprocessing.pool
import time

def myPi(m,n):
    pi = 0
    for k in range(m,n+1):
        s = 1 if k%2 else -1 
        pi += s / (2 * k - 1)
    return pi*4

if __name__ == '__main__':
    N=10000000
    with multiprocessing.pool.Pool(2) as p:
        t1=time.perf_counter()
        asr1=p.apply_async(myPi,args=(1,N//2))
        asr2=p.apply_async(myPi,args=(N//2+1,N))
        PI=asr1.get()
        PI+=asr2.get()
        t2=time.perf_counter()
    print((t2-t1)*1000)
    print(PI)    

This program runs a little faster than the version that explicitly creates its own process and provides its own locks, but the difference is small. The main advantage of using a Pool is that the process are set up by the time you want to use them.

In chapter but not in this extract

• Map_async
• Starmap_async
• Immediate Results – imap
• MapReduce
• Sharing and Locking

Summary

• A process pool, a set of processes created and ready to run, can be used to avoid the high cost of repeatedly creating processes.

• The Pool object creates a specified number of processes ready to be used.

• The apply_async method will run a function using a process from the pool.

• All of the Pool methods return an AsyncResult object which acts like what would usually be called a deferred, a promise or, in Python’s terminology, a future. This can be used to wait for a result.

• There are a range of map-like functions which allow you to run a set of functions on a set of processes from the process pool.

• map_async maps a single function onto a range of values, each run on a process from the pool.

• starmap_async works like map_async, but automates the passing of multiple parameters.

• imap works like map_async, but returns an iterator that allows you to step through results as they become available.

• The map functions are one half of a standard approach to parallel programming called MapReduce. The functools module provides a reduce function.

• You can’t pass objects like Lock and Value to a process from the process pool because the process is already running and parameters are passed as pickled data. Lock, Value and any object that has state cannot be pickled.

• To share important resources like locks you have to use the initialization parameter when the Pool is first created to supply a function which creates the shared resources in each of the processes.

Programmer's Python:
Async
Threads, processes, asyncio & more

Is now available as a print book: Amazon

Contents

1)  A Lightning Tour of Python.

2) Asynchronous Explained

3) Processed-Based Parallelism
         Extract 1 Process Based Parallism
4) Threads
         Extract 1 -- Threads
5) Locks and Deadlock

6) Synchronization

7) Sharing Data
        Extract 1 - Pipes & Queues

8) The Process Pool
        Extract 1 -The Process Pool 1 ***NEW!

9) Process Managers

10) Subprocesses

11) Futures
        Extract 1 Futures, 

12) Basic Asyncio
        Extract 1 Basic Asyncio

13) Using asyncio
        Extract 1 Asyncio Web Client
14) The Low-Level API
       Extract 1 - Streams & Web Clients
Appendix I Python in Visual Studio Code

<ASIN:1871962765>

<ASIN:1871962749>

<ASIN:1871962595>