Programmer's Python Async

Programmer's Python Async - Asyncio

Written by Mike James

Monday, 10 October 2022

Article Index
Programmer's Python Async - Asyncio
Await
Tasks

Page 3 of 3

Tasks

Tasks are coroutines plus futures. To be more exact, Task inherits from Future and keeps a reference to a coroutine that the Future is associated with. Put simply, a Task is what you add to the event loop’s queue and it is a Future plus a coroutine. The coroutine does the work and the Future returns the result.

To add a Task to the event loop you need to use the:

asyncio.create_task(coroutine, args, name = None)

This adds the coroutine to the current event loop as a Task, passing it any args you may specify and giving it a name if you specify one. The function adds the coroutine to the event loop queue ready to be executed. It doesn’t actually get to run until the main thread is free to return to the event loop and run the tasks that it finds there. This only happens when the currently executing coroutine awaits an asynchronous coroutine or terminates.

The asyncio.create_task function returns a Task which, as already explained, is a future-like object which resolves with the result when the coroutine ends. Anything that is added to the event loop has to behave like a Future as it is needed for the await to retrieve a result. For example, if we create a coroutine that prints a range of numbers then this can be added to the event loop within main:

import asyncio
async def count(n):
    for i in range(n):
        print(i)
    return n

async def main(myValue):
    t1=asyncio.create_task(count(10))  
    print("Hello Coroutine World")
    await asyncio.sleep(5)
    return myValue

result= asyncio.run(main(42))
print(result)

The count coroutine is added to the event loop before the print, but it doesn’t get to run until main awaits sleep for 5 seconds and so frees the thread. If you take out the await on sleep then the count coroutine gets to run when main finishes. If you put the await before the print then count gets to print its values before the Hello Coroutine World is displayed. When count runs depends on when main is allowed to run the event loop.

If you look at the program again you will see that count returns a result, but the result isn’t used in the program. How can we get a result from a Task? The simple answer is that we wait for its Future to resolve and for await to return its result after consuming the Future, for example:

async def main(myValue):
    t1 = asyncio.create_task(count(10))   
    print("Hello Coroutine World")
    await asyncio.sleep(5)
    result = await t1
    print(result)
    return myValue

In this case await t1 returns result which is displayed.

At this point you should be wondering why we bothered creating a Task and adding it to the event loops’s queue? Why not just use await count? That is:

async def main(myValue):
    print("Hello Coroutine World")
    await asyncio.sleep(5)
    result= await count(10)
    print(result)
    return myValue

This produces the same answer, but what happens under the hood is very different. The first version adds count to the event loop’s queue and when main sleeps for five seconds the thread is freed and t1 is allowed to execute. After the five seconds is up the await returns at once because t1 is resolved and there is a result which is displayed immediately. The second version doesn’t add count to the queue and so nothing happens while main waits for five seconds. When the wait is up the await starts count running to get the result which is then printed. In other words, when you await a Task the Task might already have run and have a result. If it hasn’t already resolved then it is taken from the queue and run just like a coroutine. That is:

when you await a coroutine it starts running to completion
when you await a Task it only starts running if it hasn’t already resolved

In both cases the thread might be released and tasks run before the coroutine or the Task completes.

You will sometimes see instructions like:

    value = await asyncio.create_task(count(10))

This adds the coroutine to the queue as a Task and then immediately awaits it, which of course, starts it running. There is no point in doing this and it is entirely equivalent to:

    value = await count(10)

In general:

await asyncio.create(coroutine)

is the same as:

await coroutine

To summarize:

asyncio.create_task(coroutine) runs the coroutine at a later time. It adds it to the event loop’s queue as a Task for execution when the thread is free
await coroutine runs the coroutine immediately
await Task runs the associated coroutine if the associated Future hasn’t ready resolved
asyncio.create_task(coroutine) is equivalent to await coroutine

Because it is often used in older examples, it is also worth mentioning,:

asyncio.ensure_future(awaitable)

This takes a coroutine, Future or any awaitable object, converts it and “ensures” that it is a future-like object and adds it to the event queue. In practice, this means that the coroutine is converted into a Task and added to the queue. In other words, it is the equivalent of create_task.

In Chapter but not included in this extract

Execution Order
Tasks and Futures
Waiting On Coroutines
Sequential and Concurrent
Canceling Tasks
Dealing With Exceptions
Shared Variables and Locks
Context Variables
Queues

Summary

The asyncio module provides single-threaded multi-tasking.
The callback is the most common way of implementing single-threaded multi-tasking but it has many disadvantages. A better method is to use a Future and an even better method is to use await.
A coroutine is a function that can be suspended and resumed by the use of the await instruction.
A coroutine has to be run in conjunction with an event loop. The asyncio.run creates an event loop and runs a coroutine using it.
A Task is a Future plus a coroutine and it is what is added to the event loop’s queue using asyncio.create_task. The Task is run when the thread becomes free.
When you await a coroutine it starts running to completion.
When you await a Task it only starts running if it isn’t already completed.
The await always returns the result of the coroutine/Task, including any exceptions that might have occurred.
If you don’t await a Task its result and any exceptions are ignored.
You can use wait_for as a version of await with a timeout.
The wait coroutine can be used to wait for the first to complete, the first to raise an exception or for all complete.
Task coroutines can be executed in sequential order by awaiting each one in turn. They can be run concurrently by adding them to the queue or by using the gather coroutine.
A Task can be canceled and this sends the CancelledError exception to the Task.
A Task returns any exceptions to the awaiting coroutine – these can be raised or processed.
Locks are less useful for coroutines because unless the thread is released they are atomic. If a race condition can occur there are asynchronous equivalents of all of the standard synchronization objects.
Shared global variables cannot be protected against race conditions using thread-local variables as only a single thread is in operation. Instead we need to use context variables.
There are asynchronous equivalents of the Thread queue objects.