Python Multithreading

How to run parallel tasks in Python using threads?

As we all know, any Python program runs as a single-threaded process. With the support threads, we can achieve the concurrency & parallelism to any Python task which can help the developer to run any task in an optimal way.
This approach can help to do any kind of a task like scraping the website, writing utilities like fetching the records, copying the object(s), sending messages, etc. This post will explain why threads are important, how to implement the threads and some tips about implementing threads.

What are Python threads? And why is it needed?

Threading is a technique by which we can achieve the parallelism in Python. By default, the threading feature is provided by the operating system.
Because it runs in the same process,  they can share the same memory space with each other. They are very lightweight & not required much overhead as compared to processes.

How to implement thread?

There are two modules supported by Python. 
  • threading (Python 3)

  • _thread (Python 2)
We are using threading for our example.

#!/usr/bin/Python 3

import threading
import time

exitFlag = 0

class CustomThread (threading.Thread):
def __init__(self, name):
threading.Thread.__init__(self)
self.name = name
def run(self):
print("Starting the thread" + self.name)
my_task(self.name, 5)
print("Exiting the thread" + self.name)
def my_task(threadName, counter):
while counter:
if exitFlag:
threadName.exit()
time.sleep(5)
print("Executing my_task by thread {} at {}".format(threadName, time.ctime()))
counter -= 1

# Create new threads
thread1 = CustomThread("Custom Thread-1")
thread2 = CustomThread("Custom Thread-2")

# Start new Threads
thread1.start() # Calling run method
thread2.start() # Calling run method
thread1.join() # Waits for to terminate
thread2.join() # Waits for to terminate
print ("Exiting from main thread")

Well, there are many ways available but we prefer to implement thread like semaphore from the threading module.

#!/usr/bin/Python 3

import threading

def my_task(semaphore, my_obj):
semaphore.acquire(blocking=False)
print("Executing my_task, performing test on {}".format(my_obj))
semaphore.release() # Release the thread

semaphore = threading.Semaphore(5) # Creating 5 threads
My_collection = [“one”, “two”, “three”, “four”, “five”, “six”, “seven”, “eight”, “nine”, “ten” ]
custom_threads = [threading.Thread(target=my_task, args=(semaphore, my_obj )) for my_obj in my_collection]
for thread in custom_threads:
thread.start()

With the support of threading, we can easily execute complicated tasks without creating process overhead.

About Author

SANDEEP NATOO

Sandeep is highly vigorous Machine learning expert with over 12+ work of experience with developing heterogeneous systems in IT sector.He is expert in building Java integrated web applications and Python data analysis stack. He has been known for translating complex datasets into meaningful insights, and his passion lies in interpreting the data and providing the valuable prediction with a good eye for detail. He is highly optimistic and avid nature, for various challenges, is his major strength.
Related Posts

Leave a Comment