Python exec command

last modified October 30, 2026 Python 3.15

In this article we show how to execute shell commands and programs in Python.

Running external programs

In Python, there are several ways to execute shell commands or programs. The subprocess module has the most powerful tools; it allows us to spawn new processes, connect to their input, output, and error pipes, and obtain their return codes. The older os.system and os.popen functions are simpler but less flexible.

Function	Description
subprocess.run	runs a command, waits for it, and returns a CompletedProcess
subprocess.check_output	runs a command and returns its output; raises an exception on failure
subprocess.Popen	starts a process and lets us interact with it while it runs
os.system	runs a command in a shell and returns its exit status
os.popen	runs a command in a shell and returns a file object connected to its output
os.exec*	replaces the current process with a new program

The os.system function

The os.system function executes a command in a subshell. The output of the command goes directly to the standard output of our program.

import os

status = os.system('echo hello from the shell')
print(f'status: {status}')

status = os.system('exit 3')
print(f'status: {status}, exit code: {os.waitstatus_to_exitcode(status)}')

The function returns the wait status of the process. On Unix, the exit code is encoded in it; the os.waitstatus_to_exitcode function extracts it.

$ python system_fun.py
hello from the shell
status: 0
status: 768, exit code: 3

The os.popen function

The os.popen function opens a pipe to or from a command. The return value is an open file object connected to the pipe, which can be read or written depending on whether the mode is 'r' (the default) or 'w'.

import os

output = os.popen('ls files').read()
print(output.split())

In the example, we read the output of the ls command for the files directory.

$ python read_output.py
['data.txt', 'names.csv', 'notes.md', 'report.pdf']

The subprocess.run function

The subprocess.run function executes a command, waits for it to complete, and returns a CompletedProcess instance. It is the recommended way to run commands.

import subprocess

result = subprocess.run(['git', '--version'], capture_output=True, text=True)

print(result)
print(f'return code: {result.returncode}')
print(f'version: {result.stdout.split()[-1]}')

The example gets the version of Git. The command is given as a list: the program name followed by its arguments.

result = subprocess.run(['git', '--version'], capture_output=True, text=True)

With capture_output=True, the standard output and the standard error output are captured in the stdout and stderr attributes. By default, they are bytes; with text=True, they are decoded into strings.

$ python version_check.py
CompletedProcess(args=['git', '--version'], returncode=0, stdout='git version 2.34.1\n', stderr='')
return code: 0
version: 2.34.1

Handling errors

A failing command does not raise an exception by default; we must check its return code.

import subprocess

result = subprocess.run(['ls', 'missing'], capture_output=True, text=True)
print(result.returncode, result.stderr.strip())

try:
    subprocess.run(['ls', 'missing'], capture_output=True, text=True, check=True)
except subprocess.CalledProcessError as e:
    print(f'command {e.cmd} failed with code {e.returncode}')

try:
    subprocess.run(['nosuchcommand'])
except FileNotFoundError as e:
    print(e)

The ls command fails with code 2 and an error message on the standard error output. With check=True, a non-zero return code raises CalledProcessError. If the program does not exist, FileNotFoundError is raised.

$ python check_errors.py
2 ls: cannot access 'missing': No such file or directory
command ['ls', 'missing'] failed with code 2
[Errno 2] No such file or directory: 'nosuchcommand'

The check_output function

The check_output function is a shortcut that returns the output of a command and raises CalledProcessError if it fails.

import subprocess

kernel = subprocess.check_output(['uname', '-s'], text=True).strip()
print(kernel)

count = subprocess.check_output(['wc', '-l', 'words.txt'], text=True)
print(count.split()[0])

We get the name of the kernel and the number of lines in the words.txt file.

$ python check_output_fun.py
Linux
7

Sending input

The input parameter sends data to the standard input of the process.

import subprocess

words = 'sky\nforest\ncup\nwood\n'

result = subprocess.run(['sort', '-r'], input=words, capture_output=True,
                        text=True)
print(result.stdout, end='')

We pass four words to the sort command, which sorts them in reverse order.

$ python send_input.py
wood
sky
forest
cup

Running commands in a shell

With shell=True, the command is a string executed by the system shell, /bin/sh on Unix. This allows shell features such as pipes, redirection, and wildcards.

import subprocess

cmd = 'sort words.txt | uniq -c | sort -rn | head -3'
result = subprocess.run(cmd, shell=True, capture_output=True, text=True)

print(result.stdout, end='')

The pipeline finds the three most common words in the file.

$ python shell_true.py
      3 sky
      2 forest
      1 wood

Shell injection

Building a shell command from untrusted data is dangerous. The shell interprets special characters such as ;, |, and $, so the data can run arbitrary commands.

import shlex
import subprocess

filename = 'words.txt; echo INJECTED'

subprocess.run(f'wc -l {filename}', shell=True)

print('---')

subprocess.run(['wc', '-l', filename])

print('---')

subprocess.run(f'wc -l {shlex.quote(filename)}', shell=True)
print(shlex.quote(filename))
print(shlex.split('grep -n "two words" file.txt'))

The file name contains a semicolon. With shell=True and an f-string, the shell runs the injected echo command. In the list form, no shell is involved and the whole string is a single argument, so wc reports a missing file. If a shell is needed, the shlex.quote function escapes the argument. The shlex.split function splits a command string into a list the way a shell would.

$ python injection.py
7 words.txt
INJECTED
---
wc: 'words.txt; echo INJECTED': No such file or directory
---
wc: 'words.txt; echo INJECTED': No such file or directory
'words.txt; echo INJECTED'
['grep', '-n', 'two words', 'file.txt']

Timeouts

The timeout parameter limits how long we wait for the command.

import subprocess
import time

start = time.perf_counter()

try:
    subprocess.run(['sleep', '10'], timeout=1)
except subprocess.TimeoutExpired as e:
    print(f'{e.cmd} timed out after {e.timeout} s')

print(f'elapsed: {time.perf_counter() - start:.1f} s')

When the timeout expires, the child process is killed and TimeoutExpired is raised.

$ python timeout_fun.py
['sleep', '10'] timed out after 1 s
elapsed: 1.0 s

Working directory and environment

The cwd parameter sets the working directory of the process and env its environment variables.

import os
import subprocess

result = subprocess.run(['ls'], cwd='files', capture_output=True, text=True)
print(result.stdout.split())

env = os.environ | {'GREETING': 'Hello there'}
cmd = ['sh', '-c', 'echo "$GREETING from $PWD"']
result = subprocess.run(cmd, cwd='files', env=env, capture_output=True,
                        text=True)
print(result.stdout, end='')

We list the files of the files directory by running ls inside it. The env mapping replaces the whole environment, so we merge our variable into a copy of os.environ with the | operator.

$ python cwd_env.py
['data.txt', 'names.csv', 'notes.md', 'report.pdf']
Hello there from /home/user/python/files

The Popen class

The process creation and management in the subprocess module is handled by the Popen class. The functions above are built on it. Popen starts the process and returns immediately, so we can interact with the process while it runs.

import subprocess

proc = subprocess.Popen(['tr', 'a-z', 'A-Z'], stdin=subprocess.PIPE,
                        stdout=subprocess.PIPE, text=True)

print(f'running: {proc.poll() is None}')

stdout, _ = proc.communicate(input='an old falcon\nan old owl\n')

print(stdout, end='')
print(f'return code: {proc.returncode}')

The poll method returns None while the process is running. The communicate method sends data to the standard input, reads the standard output and error output until the end of file, and waits for the process to terminate.

$ python communicate_fun.py
running: True
AN OLD FALCON
AN OLD OWL
return code: 0

Reading output as it arrives

The run function returns the output only after the process finishes. With Popen, we can read it line by line while the process is running.

import time

for i in range(1, 4):
    print(f'tick {i}', flush=True)
    time.sleep(0.5)

This program prints a line every half a second.

import subprocess
import sys
import time

start = time.perf_counter()

with subprocess.Popen([sys.executable, 'ticker.py'], stdout=subprocess.PIPE,
                      text=True) as proc:
    for line in proc.stdout:
        print(f'{time.perf_counter() - start:.1f} s: {line}', end='')

print(f'exit code: {proc.returncode}')

We run the ticker with the current Python interpreter, sys.executable, and iterate over its standard output. The timestamps show that each line is received when it is printed. The ticker flushes its output; a program that buffers its output would deliver all lines at the end. Used as a context manager, Popen waits for the process when the block ends.

$ python stream_output.py
0.0 s: tick 1
0.5 s: tick 2
1.0 s: tick 3
exit code: 0

Pipelines

We can connect processes into a pipeline without a shell by passing the output pipe of one process as the input of another.

import subprocess

p1 = subprocess.Popen(['sort', 'words.txt'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['uniq', '-c'], stdin=p1.stdout, stdout=subprocess.PIPE,
                      text=True)
p1.stdout.close()

output, _ = p2.communicate()
p1.wait()

for line in output.splitlines():
    count, word = line.split()
    print(f'{word}: {count}')

The example is equivalent to the sort words.txt | uniq -c shell pipeline. We close our copy of p1.stdout, so that sort receives a SIGPIPE signal if uniq exits early.

$ python pipeline.py
cup: 1
forest: 2
sky: 3
wood: 1

Running processes in parallel

Since Popen does not wait for the process, we can start several processes at once.

import subprocess
import time

start = time.perf_counter()

procs = [subprocess.Popen(['sleep', '1']) for _ in range(5)]

for proc in procs:
    proc.wait()

print(f'5 processes finished in {time.perf_counter() - start:.1f} s')

Five sleep 1 processes run at the same time, so the program finishes in about one second instead of five.

$ python parallel.py
5 processes finished in 1.0 s

Replacing the process

The os.exec* family of functions replaces the current process with a new program. The new program keeps the process ID; the Python code after the call never runs.

import os

print(f'python pid: {os.getpid()}', flush=True)

os.execvp('sh', ['sh', '-c', 'echo "shell pid: $$"'])

print('this line is never printed')

The execvp function searches for the program in the PATH and passes it the argument list, whose first item is the program name. The shell prints the same process ID as Python. We flush the output before the call, because the buffers of the replaced process are lost.

$ python replace_process.py
python pid: 139665
shell pid: 139665

Source

subprocess - Subprocess management - Python documentation

In this article we have executed shell commands and programs in Python.

Author

My name is Jan Bodnar, and I am a passionate programmer with extensive programming experience. I have been writing programming articles since 2007. To date, I have authored over 1,400 articles and 8 e-books. I possess more than ten years of experience in teaching programming.

List all Python tutorials.