Get In Touch
0% complete

Python automation and practical scripting · Lesson 3 of 3

Automate workflows with subprocesses and schedules

Coordinate external commands and scheduled jobs with logs, timeouts, and failure handling.

Lesson

1. Concept and mental model

Python can coordinate existing tools through subprocess.run(). Pass an argument list rather than building a shell command string from user input; this avoids shell parsing surprises and command injection. Set a timeout, capture output when needed, and use check=True when a nonzero exit status should raise CalledProcessError. Validate executable paths and arguments, especially when inputs come from outside the program.

2. How it works

Scheduling is usually provided by an operating-system scheduler or orchestration service, not by leaving a Python process running forever. Make a scheduled task repeatable: use stable identifiers, detect already-completed work, and define what happens if a run stops halfway. Write logs with a timestamp, outcome, and correlation identifier; send sensitive details to an approved secure store rather than standard output.

3. Use it safely and effectively

Avoid overlapping executions unless the task is designed for concurrency. Use a lock or scheduler policy when duplicate simultaneous runs could corrupt output. Choose retry behavior based on whether the operation is safe to repeat, and alert an operator after a bounded number of failures. Test subprocess failures, timeouts, missing executables, and partial output using a harmless local command before automating a production workflow.

4. Syntax reference

import subprocess

result = subprocess.run(
    ['python', '--version'],
    check=True, capture_output=True, text=True, timeout=10,
)
print(result.stdout.strip() or result.stderr.strip())

5. Guided walkthrough

Call an external program with a list of arguments, set a timeout, and decide how nonzero exit codes should be handled. Schedule the task through a managed scheduler, prevent unsafe overlapping runs, and make repeated execution safe for the same input.

6. Common mistakes to avoid

  • Building a shell command by concatenating untrusted input.
  • Retrying a non-idempotent operation without checking whether the first attempt partially succeeded.

Visual guide

Reliable scheduled automation

  1. Step 01 Trigger

    A scheduler starts one bounded run.

  2. Step 02 Execute

    Run validated commands with timeouts.

  3. Step 03 Observe

    Record status and alert on persistent failures.

Code examples

Worked example
PYTHON 
import subprocess

result = subprocess.run(
    ['python', '--version'],
    check=True, capture_output=True, text=True, timeout=10,
)
print(result.stdout.strip() or result.stderr.strip())
Expected output
Python 3.x.y (the version installed in the active environment).
Handle an expected command failure
PYTHON 
import subprocess

try:
    subprocess.run(['python', '--invalid-option'], check=True, timeout=5)
except subprocess.CalledProcessError as error:
    print(f'Exit status: {error.returncode}')
Expected output
Prints the command exit status when the command fails.
Add a timeout to a subprocess
PYTHON 
import subprocess

try:
    subprocess.run(['python', '-c', 'print(1)'], check=True, timeout=5)
    print('completed')
except subprocess.TimeoutExpired:
    print('timed out')
Expected output
completed

Key points

  • Coordinate external commands and scheduled jobs with logs, timeouts, and failure handling.
  • Prefer explicit, testable behavior over hidden side effects.
  • Validate assumptions at system boundaries and handle failure deliberately.

Check your understanding

Practice

Why pass subprocess arguments as a list?

Your learning progress

Finish this lesson