We have an API endpoint that starts a thread, and another endpoint to check the status of the thread (based on a thread ID returned by the first API call).
We use the threading module.
The function that the thread is executing may or may not sleep for a duration of time.
When we create the thread, we override the default name provided by the module and add the thread ID that was generated by us (so we can keep track).
The status endpoint gets the thread ID from the client request and simply loops over the results from threading.enumerate(). When the thread is running and not sleeping, we see that the thread is returned by the threading.enumerate() function. When it is sleeping, it is not.
The function we use to see if a thread is alive:
def thread_is_running(thread_id):
all_threads = [ t.getName() for t in threading.enumerate() ]
return any(thread_id in item for item in all_threads)
When we run in debug and print the value of "all_threads", we only see the MainThread thread during our thread's sleep time.
As soon as the sleep is over, we see our thread in the value of "all_threads".
This is how we start the thread:
thread_id = random.randint(10000, 50000)
thread_name = f"{service_name}-{thread_id}"
threading.Thread(target=drain, args=(service_name, params,), name=thread_name).start()
Is there a way to get a list of all threads including idle threads? Is a sleeping thread marked as idle? Is there a better way to pause a thread?
We thought about making the thread update it's state in a database, but due to some internal issue we currently have, we cannot 100% count on writing to our database, so we prefer checking the system for the thread's status.
Turns out the reason we did not see the thread was our use of gunicorn and multi workers.
The thread was initiated on one of the 4 configured workers while the status api call could've been handled by any of the 4 workers. only when it was handled by the worker who is also responsible of running the thread - we were able to see it in the enumerate output
Related
Folks how exactly are requests being sent from JMeter. Given this configuration does it mean that JMeter will create 200 threads, and send concurrent requests in a loop 20 times. I would assume that this means that each thread blocks until the previous request finishes, and sends the next request, and that this process is done 20 times for each thread.
Let's say I have a sample request A, num threads = 2, and loop count= 5 I imagine the workflow to be as follows
At time t0 Thread 1 -> send A to the target server
At time t0 Thread 2 -> send A to the taget server
At time t1 Thread 1 receives response, and sends A again to target server
At time t2 Thread 2 receives response and sends A again to target server
Is this the correct workflow that JMeter will follow. Please note I am using version 5.2.1
JMeter starts all the Threads defined in the Thread Group within the bounds of the Ramp-Up period
Each JMeter thread does not depend on the other threads
Once started each Thread starts executing Samplers upside down (or according to the Logic Controllers)
When there are no more samplers to execute the thread starts the next iteration
When there are no more loops to iterate - the thread is being shut down
The actual concurrency depends on test duration and application response time, it can be checked using i.e. Active Threads Over Time and Transactions Per Second listeners
You can easily track each virtual user/iteration using __threadNum() function and special ${__jm__Thread Group__idx} JMeter Variable:
I am making a bot for telegram, this bot will use a database (SQLite3).
I am familiar with threads and locks and I know that is safe to launch multiple thread that make query to the database.
My problem rises when I want to update/insert data.
With the use Condition and Event from the threading module, I can prevent new thread to access the database while a thread is updating/inserting data.
What I haven't figured out is how to wait that all the thread that are accessing the database are done, before updating/inserting data.
If I could get the count of semaphore I would just wait for it to drop to 0, but since is not possible, what approach should I use?
UPDATE: I can't use join() since I am using telegram bot and create thread dynamically with each request to my bot, therefore when a thread is created I don't know if I'll have to wait for it to end or not.
CLARIFICATION: join() can only be used if, at the start of a thread you know wether you'll have to wait for it to end or not. Since I create a thread for each request of my clients and I am unaware of what they'll ask or and when the request will be done, I can't know whether to use join() or not.
UPDATE2: Here the code regarding the locks. I haven't finished the code regarding the database since I am more concerned with the locks and it doesn't seems relevant to the question.
lock = threading.Lock()
evLock = threading.Event()
def addBehaviours(dispatcher):
evLock.set()
# (2) Fetch the list of events
events_handler = CommandHandler('events', events)
dispatcher.add_handler(events_handler)
# (3) Add a new event
addEvent_handler = CommandHandler('addEvent', addEvent)
dispatcher.add_handler(addEvent_handler)
# (2) Fetch the list of events
#run_async
def events(bot, update):
evLock.wait()
# fetchEvents()
# (3) Add a new event
#run_async
def addEvent(bot, update):
with lock:
evLock.clear()
# addEvent()
evLock.set()
You can use threading.Thread.join(). This will wait for a thread to end and only continue on when the thread is done.
Usage below:
import threading as thr
thread1 = thr.Thread() # some thread to be waited for
thread1 = thr.Thread() # something that runs after thread1 finishes
thread1.start() # start up this thread
thread1.join() # wait until this thread finishes
thread2.start()
...
i am new to Sockets programming and going through the Documentation.
From a documentation i found about CloseThreadPool() function :
CloseThreadpool function. The thread pool is closed immediately if there are no outstanding callback objects that are bound to the thread pool. If there are, then the thread pool is released asynchronously when those outstanding objects are freed.
This thread pool is in a thread itself. my main thread takes input for exit. if exit is inputted i set global variable KEEP_LISTENEING to false.
How would i wait my main thread to stop/sleep untill this function truly completes in another thread ?
Use a cleanup group to wait for all callbacks. The sequence is:
CreateThreadpoolCleanupGroup()
SetThreadpoolCallbackCleanupGroup(&CallbackEnvironment, pointerCleanup, ...)
CloseThreadpoolCleanupGroupMembers(, FALSE, )
CloseThreadpoolCleanupGroup()
I'm creating a worker thread in MFC with AfxBeginThread, but the thread is not getting scheduled. Here's the code:
CWinThread* worker = AfxBeginThread(initialUpdateWorkerThread, this);
DWORD dwExitCode = 0;
while(GetExitCodeThread(worker->m_hThread, &dwExitCode))
{
if(dwExitCode != STILL_ACTIVE)
break;
::Sleep(100);
}
When I run this, this loop just livelocks because initialUpdateWorkerThread is never called (I've put break points and message boxes at the top of it) so dwExitCode is always STILL_ACITVE. But if I put in a call to AfxMessageBox before the loop but after AfxBeginThread then the function is called. This makes me think that somehow I'm not calling the right function to get the thread scheduled, but a call to AfxMessageBox causes it to get scheduled.
How can I force the thread to be scheduled? I would think sleep would do that, but in this case it doesn't seem to.
Your worker thread is probably trying to send your main thread a message, but since you aren't processing messages on on the main thread, the worker thread simply waits. You can confirm this by simply breaking into the debugger to see what the worker thread is doing.
I have designed an application which is running 20 instance of a thread.
for(int i = 0;i<20;i++)
{
threadObj[i].start();
}
How can I wait in the main thread until those 20 threads finish?
You need to use QThread::wait().
bool QThread::wait ( unsigned long time = ULONG_MAX )
Blocks the thread until either of
these conditions is met:
The thread associated with this
QThread object has finished execution (i.e. when it returns from
run()). This function will return true if the thread has finished. It
also returns true if the thread has
not been started yet.
time milliseconds has elapsed. If time is
ULONG_MAX (the default), then the wait
till never timeout (the thread must
return from run()). This function
will return false if the wait timed
out.
This provides similar functionality to
the POSIX pthread_join() function.
Just loop over the threads and call wait() for each one.
for(int i = 0;i < 20;i++)
{
threadObj[i].wait();
}
If you want to let the main loop run while you're waiting. (E.g. to process events and avoid rendering the application unresponsible.) You can use the signals & slots of the threads. QThread's got a finished() singal which you can connect to a slot that remembers which threads have finished yet.
You can also use QWaitCondition
What Georg has said is correct. Also remember you can call signal slot from across threads. So you can have your threads emit a signal to you upon completion. SO you can keep track of no of threads that have completed their tasks/have exited. This could be useful if you don't want your Main thread to go in a blocking call wait.