Tutorial Playlist
200 Lessons1. Introduction to Python
2. Features of Python
3. How to install python in windows
4. How to Install Python on macOS
5. Install Python on Linux
6. Hello World Program in Python
7. Python Variables
8. Global Variable in Python
9. Python Keywords and Identifiers
10. Assert Keyword in Python
11. Comments in Python
12. Escape Sequence in Python
13. Print In Python
14. Python-if-else-statement
15. Python for Loop
16. Nested for loop in Python
17. While Loop in Python
18. Python’s do-while Loop
19. Break in Python
20. Break Pass and Continue Statement in Python
21. Python Try Except
22. Data Types in Python
23. Float in Python
24. String Methods Python
25. List in Python
26. List Methods in Python
27. Tuples in Python
28. Dictionary in Python
29. Set in Python
30. Operators in Python
31. Boolean Operators in Python
32. Arithmetic Operators in Python
33. Assignment Operator in Python
34. Bitwise operators in Python
35. Identity Operator in Python
36. Operator Precedence in Python
37. Functions in Python
38. Lambda and Anonymous Function in Python
39. Range Function in Python
40. len() Function in Python
41. How to Use Lambda Functions in Python?
42. Random Function in Python
43. Python __init__() Function
44. String Split function in Python
45. Round function in Python
46. Find Function in Python
47. How to Call a Function in Python?
48. Python Functions Scope
49. Method Overloading in Python
50. Method Overriding in Python
51. Static Method in Python
52. Python List Index Method
53. Python Modules
54. Math Module in Python
55. Module and Package in Python
56. OS module in Python
57. Python Packages
58. OOPs Concepts in Python
59. Class in Python
60. Abstract Class in Python
61. Object in Python
62. Constructor in Python
63. Inheritance in Python
64. Multiple Inheritance in Python
65. Encapsulation in Python
66. Data Abstraction in Python
67. Opening and closing files in Python
68. How to open JSON file in Python
69. Read CSV Files in Python
70. How to Read a File in Python
71. How to Open a File in Python?
72. Python Write to File
73. JSON Python
74. Python JSON – How to Convert a String to JSON
75. Python JSON Encoding and Decoding
76. Exception Handling in Python
77. Recursion in Python
78. Python Decorators
79. Python Threading
80. Multithreading in Python
81. Multiprocеssing in Python
82. Python Regular Expressions
83. Enumerate() in Python
84. Map in Python
85. Filter in Python
86. Eval in Python
87. Difference Between List, Tuple, Set, and Dictionary in Python
88. List to String in Python
89. Linked List in Python
90. Length of list in Python
91. Reverse a List in Python
92. Python List remove() Method
93. How to Add Elements in a List in Python
94. How to Reverse a List in Python?
95. Difference Between List and Tuple in Python
96. List Slicing in Python
97. Sort in Python
98. Merge Sort in Python
99. Selection Sort in Python
100. Sort Array in Python
101. Sort Dictionary by Value in Python
102. Datetime Python
103. Random Number in Python
104. 2D Array in Python
105. Abs in Python
106. Advantages of Python
107. Anagram Program in Python
108. Append in Python
109. Applications of Python
110. Armstrong Number in Python
111. Assert in Python
112. Binary Search in Python
113. Binary to Decimal in Python
114. Bool in Python
115. Calculator Program in Python
116. chr in Python
117. Control Flow Statements in Python
118. Convert String to Datetime Python
119. Count in python
120. Counter in Python
121. Data Visualization in Python
122. Datetime in Python
123. Extend in Python
124. F-string in Python
125. Fibonacci Series in Python
126. Format in Python
127. GCD of Two Numbers in Python
128. How to Become a Python Developer
129. How to Run Python Program
130. In Which Year Was the Python Language Developed?
131. Indentation in Python
132. Index in Python
133. Interface in Python
134. Is Python Case Sensitive?
135. Isalpha in Python
136. Isinstance() in Python
137. Iterator in Python
138. Join in Python
139. Leap Year Program in Python
140. Lexicographical Order in Python
141. Literals in Python
142. Matplotlib
143. Matrix Multiplication in Python
144. Memory Management in Python
145. Modulus in Python
146. Mutable and Immutable in Python
147. Namespace and Scope in Python
148. OpenCV Python
149. Operator Overloading in Python
150. ord in Python
151. Palindrome in Python
152. Pass in Python
153. Pattern Program in Python
154. Perfect Number in Python
155. Permutation and Combination in Python
156. Prime Number Program in Python
157. Python Arrays
158. Python Automation Projects Ideas
159. Python Frameworks
160. Python Graphical User Interface GUI
161. Python IDE
162. Python input and output
163. Python Installation on Windows
164. Python Object-Oriented Programming
165. Python PIP
166. Python Seaborn
167. Python Slicing
168. type() function in Python
169. Queue in Python
170. Replace in Python
171. Reverse a Number in Python
172. Reverse a string in Python
173. Reverse String in Python
174. Stack in Python
175. scikit-learn
176. Selenium with Python
177. Self in Python
178. Sleep in Python
179. Speech Recognition in Python
180. Split in Python
181. Square Root in Python
182. String Comparison in Python
183. String Formatting in Python
184. String Slicing in Python
185. Strip in Python
186. Subprocess in Python
Now Reading
187. Substring in Python
188. Sum of Digits of a Number in Python
189. Sum of n Natural Numbers in Python
190. Sum of Prime Numbers in Python
191. Switch Case in Python
192. Python Program to Transpose a Matrix
193. Type Casting in Python
194. What are Lists in Python?
195. Ways to Define a Block of Code
196. What is Pygame
197. Why Python is Interpreted Language?
198. XOR in Python
199. Yield in Python
200. Zip in Python
In this tutorial, we delve deep into one of Python’s most powerful modules: subprocess. Designed to spawn new processes, connect to their input/output/error pipes, and obtain their return codes, the subprocess module is instrumental for professionals seeking advanced Python functionality. By mastering subprocess in Python, developers gain a formidable toolset to execute and communicate with system commands directly through Python scripts.
Understanding subprocess in Python is pivotal for professionals eager to execute shell commands and interact with other software components directly from Python scripts. This module bridges the gap between Python and external commands, offering the capability to run these commands, communicate with them, and retrieve their outputs seamlessly. This tutorial will explore key functionalities, contrasting methods, and best practices, ensuring an enriched comprehension.
In Python, a process refers to a separate and independent instance of a running program. Each process has its memory space, resources, and execution environment. Processes are fundamental for multitasking, allowing your Python program to run multiple tasks concurrently.
To start a new process in Python, you can use the multiprocessing module, which provides a high-level interface for creating and managing processes. Here's an overview of how to start a process:
1. Import the multiprocessing module: You can use the syntax below to import the multiprocessing module.
Syntax:
import multiprocessing
2. Define a target function: Create a Python function that represents the task you want to run in the new process.
Syntax:
def my_function(arg1, arg2):
# Your task logic here
pass
3. Create a Process object: Instantiate a Process object and provide the target function and its arguments as arguments to the constructor.
Syntax:
my_process = multiprocessing.Process(target=my_function, args=(arg1_value, arg2_value))
4. Start the process: Use the start() method to launch the new process.
Syntax:
my_process.start()
Optionally, you can wait for the process to finish. We can use the join() method to wait for the new process to complete its execution.
Syntax:
my_process.join()
Here's a basic example of starting a simple process:
import multiprocessing
def print_numbers():
for i in range(1, 6):
print(f"Number {i}")
if __name__ == "__main__":
# Create a process that prints numbers
number_process = multiprocessing.Process(target=print_numbers)
# Start the process
number_process.start()
# Wait for the process to finish
number_process.join()
print("Main program finished")
In this example, we define a target function print_numbers that prints numbers from 1 to 5. We create a new process, number_process, and start it. The main program waits for the process to finish using join().
The subprocess.call() function in Python is a part of the subprocess module, and it is used to run external shell commands or applications from within a Python script. It allows you to interact with the operating system's shell, execute commands, and capture their output or return codes. subprocess.call() is a convenient way to execute external programs and manage their execution.
The subprocess.call() function is a versatile tool for running external programs and shell commands from your Python scripts. It's commonly used for tasks like running system utilities, interacting with other applications, or automating tasks that involve executing command-line tools. However, we have to be cautious when using subprocess.call() with user-generated input to avoid security vulnerabilities such as shell injection attacks.
Here's the basic syntax of subprocess.call():
subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
In the above syntax,
Here's a simple example of using subprocess.call() to run the ls command on a Unix-based system:
import subprocess
# Run the "ls" command
subprocess.call(["ls", "-l"])
In this example, we provide the command and its arguments as a list. subprocess.call() executes the command, and the output is displayed in the console.
You can also capture the command's output by redirecting stdout like this:
import subprocess
# Run the "ls" command and capture its output
output = subprocess.check_output(["ls", "-l"], universal_newlines=True)
print(output)
In this case, the universal_newlines=True argument makes sure that the output is returned as a string.
Saving process output (stdout) in the subprocess module in Python allows you to capture and manipulate the standard output (stdout) generated by an external command or process. You can then process this output within your Python script, making it a powerful tool for automation and data processing tasks.
Here's an example:
import subprocess
# Define the command to list all files in the current directory
command = ["ls", "-l"]
# Run the command and capture stdout as a byte stream
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
# Read the stdout and stderr byte streams
stdout, stderr = process.communicate()
# Check the return code of the command
return_code = process.returncode
# Process the stdout
if return_code == 0:
# Split the stdout into lines
lines = stdout.splitlines()
# Print the first few lines
print("First few lines of directory listing:")
for line in lines[:5]: # Display the first 5 lines
print(line)
# Count the number of files and directories
num_files = len(lines) - 1 # Subtract 1 for the header line
print(f"Total number of files and directories: {num_files}")
else:
# Handle errors
print(f"Error executing command: {stderr}")
# Optionally, you can also process the stderr if needed
if stderr:
print(f"Error output: {stderr}")
In the above example, we import the subprocess module to work with external commands. After that, we define the command variable as a list representing the command to list all files in the current directory, ["ls", "-l"].
We use subprocess.Popen to run the command and then specify stdout=subprocess.PIPE to capture stdout as a byte stream and stderr=subprocess.PIPE to capture stderr if there are any errors. The text=True argument indicates that we want to work with text data instead of byte data. Then, we use process.communicate() to read the stdout and stderr byte streams. This function waits for the command to complete and returns both stdout and stderr as strings.
We check the return code of the command using process.returncode. A return code of 0 typically indicates success. If the return code is 0, we split the stdout into lines and print the first few lines of the directory listing. We also count the number of files and directories by subtracting one for the header line.
If there's an error, we handle it by printing the error message from stderr. Optionally, we can process the stderr if needed.
In Python, the subprocess module is recommended over older methods like os.system(), os.spawn*(), os.popen*(), and commands.*() for running external commands and managing processes. These older methods are now considered deprecated or less efficient compared to subprocess.
Here's a brief overview of these older methods and their replacement Python subprocess examples:
os.system() is used to run a shell command as if it were run in a terminal. It returns the exit status of the command.
Example:
import os
# Deprecated: Run a shell command using os.system()
exit_status = os.system("ls -l")
Replacement with subprocess:
import subprocess
# Recommended: Use subprocess.run() to run a command
result = subprocess.run(["ls", "-l"], stdout=subprocess.PIPE, text=True)
The os.spawn*() functions are used to spawn a new process. They are lower-level and less portable than subprocess.
Example:
import os
# Deprecated: Use os.spawn*() to create a new process
pid = os.spawnlp(os.P_WAIT, "ls", "ls", "-l")
Replacement with subprocess:
import subprocess
# Recommended: Use subprocess.Popen() to start a new process
process = subprocess.Popen(["ls", "-l"], stdout=subprocess.PIPE, text=True)
The os.popen*() functions are used to open a pipe to a command's stdout or stderr. They are less flexible than subprocess.
Example:
import os
# Deprecated: Use os.popen*() to open a pipe to a command
pipe = os.popen("ls -l")
Replacement with subprocess:
import subprocess
# Recommended: Use subprocess.Popen() with stdout or stderr pipes
process = subprocess.Popen(["ls", "-l"], stdout=subprocess.PIPE, text=True)
The commands module was used to run shell commands and capture their output.
It is deprecated and should not be used in modern Python.
Example:
import commands
# Run a command and capture its output
output = commands.getoutput("ls -l")
Replacement with subprocess:
import subprocess
# Run a command and capture its output
output = subprocess.check_output(["ls", "-l"], text=True)
Harnessing the capabilities of the subprocess module, developers are equipped to integrate Python with myriad external processes, expanding its utility and scope. As professionals in the tech realm, continuous learning is paramount. While this tutorial offers a primer into subprocess, there’s always more to explore. upGrad provides diverse courses that cater to the evolving needs of tech professionals, serving as a beacon for those looking to upskill and elevate their career trajectories.
1. Why might one opt for subprocess.run over other methods?
The subprocess.run Python method encapsulates simplicity and efficiency, making it suitable for modern Python applications that require straightforward command execution.
2. How do Python subprocess Popen vs. run differ?
subprocess.Popen provides a granular approach, affording more control over the spawned process, while subprocess.run offers a direct method for command execution.
3. When should I use Python subprocess shell=true?
Use shell=true for direct shell command execution, but ensure untrusted input is stringently validated to prevent potential security risks.
4. What is the role of subprocess.poll?
The subprocess.poll method offers a non-blocking way to check the termination status of a spawned process.
5. How can I make my subprocess call wait until the command completes?
Use the subprocess Popen wait method to ensure sequential execution, waiting for a command to conclude before proceeding.
PAVAN VADAPALLI
Director of Engineering
Director of Engineering @ upGrad. Motivated to leverage technology to solve problems. Seasoned leader for startups and fast moving orgs. Working …Read More
Popular
Talk to our experts. We’re available 24/7.
Indian Nationals
1800 210 2020
Foreign Nationals
+918045604032
upGrad does not grant credit; credits are granted, accepted or transferred at the sole discretion of the relevant educational institution offering the diploma or degree. We advise you to enquire further regarding the suitability of this program for your academic, professional requirements and job prospects before enrolling. upGrad does not make any representations regarding the recognition or equivalence of the credits or credentials awarded, unless otherwise expressly stated. Success depends on individual qualifications, experience, and efforts in seeking employment.
upGrad does not grant credit; credits are granted, accepted or transferred at the sole discretion of the relevant educational institution offering the diploma or degree. We advise you to enquire further regarding the suitability of this program for your academic, professional requirements and job prospects before enr...