The complete program is:

import subprocess
import threading
class AsyncReader():
    def __init__(self,stream):
        self._stream=stream
        self._char=""
        self.t=threading.Thread(target=self._get,
                                           daemon=True)
        self.t.start()
    def get(self):       
        self.t.join(0.04)
        if self.t.is_alive():
            return ""
        else:
            result=self._char
            self.t=threading.Thread(target=self._get,
                                           daemon=True)
            self.t.start()
            return result
            
    def _get(self):
        self._char=self._stream.read(1)
    
    def readmessage(self):
        ans=""
        while True:
            char=self.get()
            if char=="":
                break
            ans+=char
        return ans
p = subprocess.Popen(["myscript.bat"],
                 stdout=subprocess.PIPE,
                 stdin=subprocess.PIPE,
                 bufsize=1,universal_newlines=True,
                 text=True)
aRead=AsyncReader(p.stdout)
ans=aRead.readmessage()
print("message",ans)
p.stdin.write("mike\n")
ans=aRead.readmessage()
print("message",ans)

This is not an ideal implementation, but the idea is sound. In particular, it uses the state of the thread to signal whether or not there is a character ready to read and this means creating and destroying a thread per character. This is inefficient, but in this application this doesn’t matter too much – user I/O is slow. A better solution would be to use a thread pool, but an even better solution is to use a queue to store characters as they are read. This introduces another layer of buffering to the I/O, but at least it is one that is under your control.

Using subprocess

The idea of using Python to control another program written in another language is an attractive one, but it is difficult to make work. As already mentioned, many programs don’t use stdin and stdout even though they are command line, character-based applications. Sometimes this is for security reasons – typing in passwords is the most common application. Sometimes this is for flexibility – text editors need to control the screen layout in sophisticated ways. Even if a program does use the standard I/O streams it can still be difficult to get right even with the use of buffering and the assumption that a flexible human will be reading and interpreting what to do next. The only way to find out if a program can be automated is to try it.

You will also find that there are a great many parameters that apply to Popen that we haven’t covered. The documentation is often vague about what they do and they tend to be system-dependent. This is yet another challenge to getting a subprocess to work. This said, when you can make it work it can be a shortcut that makes something possible.

It is worth knowing that asyncio also provides facilities to work with subprocesses and these fit in with the wider asynchronous approach. asyncio also seems to be free of the buffering problems we encounter using a pipe.

Summary

• The subprocess module is designed to allow you to run other programs under the control of Python. This is not an easy task given that programs are generally designed to work with a human user.

• The run function is the simple way to run a program and interact with it.

• The run function blocks until the program has finished and then returns any output it might have produced.

• The run function is easy to use but restricts how you can interact with the program. The Popen function returns immediately and lets you interact with the running program as if you were the user.

• A big problem is that I/O between the program and Popen is via a buffered pipe. To avoid deadlock when reading from a buffered pipe the communicate method is provided.

• A better approach to working with a buffered pipe is to provide a non-blocking read which can be done with the help of another thread.

• Even if you have mastered the art of interaction with another program, it can still be difficult to do the job reliably. Humans interact with programs in subtle ways and replicating this using a Python program can be very difficult.

• The only way to discover if you can automate the use of a program is to actually try it.

Programmer's Python:
Async
Threads, processes, asyncio & more

Is now available as a print book: Amazon

Contents

1)  A Lightning Tour of Python.

2) Asynchronous Explained

3) Processed-Based Parallelism
         Extract 1 Process Based Parallism
4) Threads
         Extract 1 -- Threads
5) Locks and Deadlock

6) Synchronization

7) Sharing Data
        Extract 1 - Pipes & Queues

8) The Process Pool
        Extract 1 -The Process Pool 1 

9) Process Managers

10) Subprocesses ***NEW!

11) Futures
        Extract 1 Futures, 

12) Basic Asyncio
        Extract 1 Basic Asyncio

13) Using asyncio
        Extract 1 Asyncio Web Client
14) The Low-Level API
       Extract 1 - Streams & Web Clients
Appendix I Python in Visual Studio Code

<ASIN:1871962765>

<ASIN:1871962749>

<ASIN:1871962595>