Hystrix thread-isolation based on Group Key or Command name? - semaphore

I noticed that Hystrix has two thread-isolation strategy: Thread and Semaphore.
By default Hystrix used Thread strategy and controls it by hystrix.threadpool.default.coreSize and because of ca ommand with same group key will use the same thread pool. So it is based on group key.
When Hystrix is using Semaphore strategy, the Semaphore will save in a ConcurrentHashMap and the key is command name, will it be based on command name?
Here is the code:
/**
* Get the TryableSemaphore this HystrixCommand should use for execution if not running in a separate thread.
*
* #return TryableSemaphore
*/
protected TryableSemaphore getExecutionSemaphore() {
if (properties.executionIsolationStrategy().get() == ExecutionIsolationStrategy.SEMAPHORE) {
if (executionSemaphoreOverride == null) {
TryableSemaphore _s = executionSemaphorePerCircuit.get(commandKey.name());
if (_s == null) {
// we didn't find one cache so setup
executionSemaphorePerCircuit.putIfAbsent(commandKey.name(), new TryableSemaphoreActual(properties.executionIsolationSemaphoreMaxConcurrentRequests()));
// assign whatever got set (this or another thread)
return executionSemaphorePerCircuit.get(commandKey.name());
} else {
return _s;
}
} else {
return executionSemaphoreOverride;
}
} else {
// return NoOp implementation since we're not using SEMAPHORE isolation
return TryableSemaphoreNoOp.DEFAULT;
}
}
Why they have difference scope? I write some testing code to prove it:
public static void main(String[] args) throws InterruptedException {
int i = 0;
while (i++ < 20) {
final int index = i;
new Thread(() -> {
System.out.println(new ThreadIsolationCommand(index).execute());
System.out.println(new SemaphoreIsolationCommand(index).execute());
}).start();
}
}
static class ThreadIsolationCommand extends HystrixCommand<String> {
private int index;
protected ThreadIsolationCommand(int index) {
super(
Setter
.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ThreadIsolationCommandGroup"))
.andCommandKey(HystrixCommandKey.Factory.asKey(String.valueOf(index)))
.andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
.withExecutionIsolationStrategy(HystrixCommandProperties.ExecutionIsolationStrategy.THREAD)
)
.andThreadPoolPropertiesDefaults(HystrixThreadPoolProperties.Setter()
.withCoreSize(10)
)
);
this.index = index;
}
#Override
protected String run() throws Exception {
Thread.sleep(500);
return "Hello Thread " + index;
}
#Override
protected String getFallback() {
return "Fallback Thread " + index;
}
}
static class SemaphoreIsolationCommand extends HystrixCommand<String> {
private int index;
protected SemaphoreIsolationCommand(int index) {
super(
Setter
.withGroupKey(HystrixCommandGroupKey.Factory.asKey("SemaphoreIsolationCommandGroup"))
.andCommandKey(HystrixCommandKey.Factory.asKey(String.valueOf(index)))
.andCommandPropertiesDefaults(HystrixCommandProperties.Setter()
.withExecutionIsolationStrategy(HystrixCommandProperties.ExecutionIsolationStrategy.SEMAPHORE)
.withExecutionIsolationSemaphoreMaxConcurrentRequests(10)
)
);
this.index = index;
}
#Override
protected String run() throws Exception {
Thread.sleep(500);
return "Hello Semaphore " + index;
}
#Override
protected String getFallback() {
return "Fallback Semaphore " + index;
}
}
This command with same group key and different name, result is :
Fallback Thread 9
Fallback Thread 1
Fallback Thread 8
Fallback Thread 19
Fallback Thread 20
Fallback Thread 14
Fallback Thread 3
Fallback Thread 13
Fallback Thread 17
Fallback Thread 10
Hello Thread 5
Hello Semaphore 17
Hello Semaphore 14
Hello Thread 2
Hello Semaphore 3
Hello Thread 7
Hello Thread 15
Hello Thread 4
Hello Semaphore 13
Hello Semaphore 1
Hello Thread 11
Hello Semaphore 20
Hello Semaphore 19
Hello Thread 18
Hello Thread 12
Hello Semaphore 8
Hello Semaphore 9
Hello Thread 6
Hello Thread 16
Hello Semaphore 10
Hello Semaphore 5
Hello Semaphore 2
Hello Semaphore 7
Hello Semaphore 15
Hello Semaphore 4
Hello Semaphore 11
Hello Semaphore 12
Hello Semaphore 6
Hello Semaphore 18
Hello Semaphore 16
Only Thread strategy has failed. Is that right?

Related

How to work volatile with multithread java

Volatile provides reading and writing to program memory bypassing cache. We should use volatile when there is a common resource, such as an integer variable
So, I did a little experiment.
First I had the following classes:
public class Main3 {
static int j = 0;
public static void main(String[] args) throws InterruptedException {
new Thread1().start();
new Thread2().start();
}
static int i = 0;
static class Thread1 extends Thread{
#Override
public void run() {
while(i<5){
System.out.println("thread1 i = "+i);
i++;
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
static class Thread2 extends Thread{
#Override
public void run() {
int val = i;
while(val<5){
if(val!=i) {
System.out.println("thread2 i = " + i);
val = i;
}
}
}
}
}
In the first thread, the variable changes its value. The second thread monitors the change and displays the text on the console if it happened
i variable is a shared resource. And without volatile the program prints what was expected and cant :
thread1 i = 0
thread1 i = 1
thread1 i = 2
thread1 i = 3
thread1 i = 4
But, if i make this (add sleep to Thread2 class):
static class Thread2 extends Thread{
#Override
public void run() {
int val = i;
while(val<5){
if(val!=i) {
System.out.println("thread2 i = " + i);
val = i;
}
try {
sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
the program prints this and successfully finished:
thread1 i = 0
thread2 i = 1
thread1 i = 1
thread2 i = 2
thread1 i = 2
thread2 i = 3
thread1 i = 3
thread2 i = 4
thread1 i = 4
thread2 i = 5
So, if threads cache values, then why does sleep caching disappear with the advent of sleep?

Reorder boost handlers after post

I would like to reorder the handlers processed by a boost io_service:
This is my pseudocode:
start()
{
io.run();
}
thread1()
{
io.post(myhandler1);
}
thread2()
{
io.post(myhandler2);
}
thread1() and thread2() are called independently.
In this case, the io_service processes the handler in the post order.
Queue example: myhandler1|myhandler1|myhandler2|myhandler1|myhandler2
How to modify the io_service processing order to execute myhandler1 and myhandler2 one after the other ?
New Queue example: myhandler1|myhandler2|myhandler1|myhandler2|myhandler1
I wrote this code but CPU usage is 100%:
start()
{
while(1)
{
io1.poll_one();
io2.poll_one();
}
}
thread1()
{
io1.post(myhandler1);
}
thread2()
{
io2.post(myhandler2);
}
Thanks
I'd use two queues. From this ASIO anwer I made once (Non blocking boost io_service for deadline_timers) I took the thread_pool class.
I split it into task_queue and thread_pool classes.
I created a worker type that knows how to juggle two queues:
struct worker {
task_queue q1, q2;
void wake() {
q1.wake();
q2.wake();
}
void operator()(boost::atomic_bool& shutdown) {
std::cout << "Worker start\n";
while (true) {
auto job1 = q1.dequeue(shutdown);
if (job1) (*job1)();
auto job2 = q2.dequeue(shutdown);
if (job2) (*job2)();
if (shutdown && !(job1 || job2))
break;
}
std::cout << "Worker exit\n";
}
};
You can see how the worker loop is structured so that - if tasks are enqueued - queues will be served in alternation.
Note: the wake() call is there for reliable shutdown; the queues use blocking waits, and hence they will need to be signaled (woken up) when the shutdown flag is toggled.
Full Demo
Live On Coliru
#include <boost/function.hpp>
#include <boost/optional.hpp>
#include <boost/thread.hpp>
#include <boost/atomic.hpp>
#include <iostream>
#include <deque>
namespace custom {
using namespace boost;
class task_queue {
private:
mutex mx;
condition_variable cv;
typedef function<void()> job_t;
std::deque<job_t> _queue;
public:
void enqueue(job_t job)
{
lock_guard<mutex> lk(mx);
_queue.push_back(job);
cv.notify_one();
}
template <typename T>
optional<job_t> dequeue(T& shutdown)
{
unique_lock<mutex> lk(mx);
cv.wait(lk, [&] { return shutdown || !_queue.empty(); });
if (_queue.empty())
return none;
job_t job = _queue.front();
_queue.pop_front();
return job;
}
void wake() {
lock_guard<mutex> lk(mx);
cv.notify_all();
}
};
template <typename Worker> class thread_pool
{
private:
thread_group _pool;
boost::atomic_bool _shutdown { false };
Worker _worker;
void start() {
for (unsigned i = 0; i < 1 /*boost::thread::hardware_concurrency()*/; ++i){
std::cout << "Creating thread " << i << "\n";
_pool.create_thread([&] { _worker(_shutdown); });
}
}
public:
thread_pool() { start(); }
~thread_pool() {
std::cout << "Pool going down\n";
_shutdown = true;
_worker.wake();
_pool.join_all();
}
Worker& get_worker() { return _worker; }
};
struct worker {
task_queue q1, q2;
void wake() {
q1.wake();
q2.wake();
}
void operator()(boost::atomic_bool& shutdown) {
std::cout << "Worker start\n";
while (true) {
auto job1 = q1.dequeue(shutdown);
if (job1) (*job1)();
auto job2 = q2.dequeue(shutdown);
if (job2) (*job2)();
if (shutdown && !(job1 || job2))
break;
}
std::cout << "Worker exit\n";
}
};
}
void croak(char const* queue, int i) {
static boost::mutex cout_mx;
boost::lock_guard<boost::mutex> lk(cout_mx);
std::cout << "thread " << boost::this_thread::get_id() << " " << queue << " task " << i << "\n";
}
int main() {
custom::thread_pool<custom::worker> pool;
auto& queues = pool.get_worker();
for (int i = 1; i <= 10; ++i) queues.q1.enqueue([i] { croak("q1", i); });
for (int i = 1; i <= 10; ++i) queues.q2.enqueue([i] { croak("q2", i); });
}
Prints e.g.
Creating thread 0
Pool going down
Worker start
thread 7f7311397700 q1 task 1
thread 7f7311397700 q2 task 1
thread 7f7311397700 q1 task 2
thread 7f7311397700 q2 task 2
thread 7f7311397700 q1 task 3
thread 7f7311397700 q2 task 3
thread 7f7311397700 q1 task 4
thread 7f7311397700 q2 task 4
thread 7f7311397700 q1 task 5
thread 7f7311397700 q2 task 5
thread 7f7311397700 q1 task 6
thread 7f7311397700 q2 task 6
thread 7f7311397700 q1 task 7
thread 7f7311397700 q2 task 7
thread 7f7311397700 q1 task 8
thread 7f7311397700 q2 task 8
thread 7f7311397700 q1 task 9
thread 7f7311397700 q2 task 9
thread 7f7311397700 q1 task 10
thread 7f7311397700 q2 task 10
Worker exit
Generalizing it
Here it is generalized for more queues (e.g. three):
Live On Coliru
Note that the above have 1 worker thread servicing; if you created more than 1 thread, each thread individually would alternate between queues, but overall the order would be undefined (because the thread scheduling is undefined).
The generalized version is somewhat more accurate here since it shared the idx variable between worker threads, but the actual output order still depends on thread scheduling.
Using run_one() instead of poll_one() should work (note that reset() is also required):
start()
{
while(1)
{
io1.run_one();
io2.run_one();
io1.reset();
io2.reset();
}
}
However, I don't know if this is a good solution to any actual problem you might have. This is one of those cases where the question, "What are you really trying to do?" seems relevant. For example, if it makes sense to run handler2 after every invocation of handler1, then perhaps handler1 should invoke handler2.

How to send signal from Singleton thread to another thread (Not singleton)

I'm facing a problem while creating a Singleton class with it's own thread that sends signal to another thread which is not a singleton class.
Consumer.h
class Consumer : public QThread
{
Q_OBJECT
public:
explicit Consumer(QObject *parent = 0);
Consumer(Worker *Worker);
signals:
void toMessage(const bool& keepCycle);
public slots:
void getMessage(const QString& str);
private:
int m_counter;
};
Consumer.cpp
Consumer::Consumer(QObject *parent) :
QThread(parent)
{
m_counter = 0;
connect(Worker::Instance(), SIGNAL(sendMessage(QString)), this, SLOT(getMessage(QString)));
connect(this, SIGNAL(toMessage(bool)), Worker::Instance(), SLOT(fromMessage(bool)));
}
// Get's message from Singleton thread if counter > 5 sends signal to terminate cycle in Singleton thread
void Consumer::getMessage(const QString &str)
{
m_counter++;
if(m_counter <= 5) {
qDebug() << "Got message " << m_counter << ": " << str << "\n";
return;
}
else {
emit toMessage(false);
}
}
Singleton is done as follows (suspect it's Not Thread-safe):
template <class T>
class Singleton
{
public:
static T* Instance()
{
if(!m_Instance) m_Instance = new T;
assert(m_Instance != NULL);
return m_Instance;
}
protected:
Singleton();
~Singleton();
private:
Singleton(Singleton const&);
Singleton& operator=(Singleton const&);
static T* m_Instance;
};
template <class T> T* Singleton<T>::m_Instance = NULL;
And Worker Singleton class
class Worker : public QThread
{
Q_OBJECT
public:
explicit Worker(QObject *parent = 0);
void run();
signals:
void sendMessage(const QString& str);
public slots:
void fromMessage(const bool& keepCycle);
private:
volatile bool m_keepCycle;
};
typedef Singleton<Worker> Worker;
Worker.cpp
Worker::Worker(QObject *parent) :
QThread(parent)
{
m_keepCycle = true;
}
void Worker::run()
{
while(true) {
if(m_keepCycle) {
QString str = "What's up?";
ElWorker::Instance()->sendMessage(str);
}
else {
qDebug() << "Keep Alive" << false;
break;
}
}
qDebug() << "Value of keepCycle" << m_keepCycle;
}
void Worker::fromMessage(const bool &keepCycle)
{
m_keepCycle = keepCycle;
qDebug() << "\nMessage FROM: " << keepCycle << "\n";
}
The main.cpp
Consumer consumer;
ElWorker::Instance()->start();
consumer.start();
Can you help me to create thread-safe Singleton and to send signals between threads?
First of all, it is highly recommended to separate worker from it's thread:
class Object : public QObject
{
...
public slots:
void onStarted(); // if needed
void onFinished(); // if needed
...
};
...
mObject = QSharedPointer < Object >(new Object);
mThread = new QThread(this);
mObject->moveToThread(mThread);
connect(mThread, SIGNAL(started()), mObject, SLOT(onStarted())); // if needed
connect(mThread, SIGNAL(finished()), mObject, SLOT(onFinished())); // if needed
mThread->start();
Second of all, there are a lot of ways of creating a singleton. My favourite is this:
Object * obj(QObject *parent = 0)
{
static Object *mObj = new Object(parent);
return mObj;
}
...
obj(this); // creating
obj()->doStuff(); // using
Now, about thread-safety. Sending signals is thread-safe, unless you're sending pointers or non-constant references. Which, according to your code, you are not. So, you should be fine.
UPDATE
Actually, I didn't get how created thread-safe singleton above and I'm
sending a signal from Worker TO Consumer Not a Thread itself? – hiken
Static values inside of function are created and initialized only once, so the first time you call obj function mObj is created and returned and each other time you call it, previously created mObj is returned. Also, I didn't say, it's thread-safe, all I said - I like this way better, then template one, because:
it is simplier
requires less code
works with QObject without problems
Yes, you should send signals from worker class, not thread one. Qt's help has a good example (the first one, not the second one): http://doc.qt.io/qt-5/qthread.html#details. The only thing QThread should be used for - is controlling thread's flow. There are some situations, when you need to derive from QThread and rewrite QThread::run, but your case isn't one of them.

Semaphore not working as expected ,it is getting locked

i am using semaphore for thread communication i have two threads one is OddThread and another is EvenThread ,i am printing value from 1 to 10 OddThread will print only odd numbers between 1 to 10 and EvenThread thread is printing only even numbers between 1 to 10. for that i have used semaphore for threads to communicate properly .what is actually happening is that OddThread is printing only 1 and EvenThread only 2 and then both get stopped. I am not under standing what is actually happening.can any body suggest.
public class ThreadProducerConsumerSemaphore {
/**
* #param args
*/
public static void main(String[] args) {
Semaphore p = new Semaphore(1);
Semaphore c = new Semaphore(0);
OddThread producer = new OddThread(p, c);
EvenThread consumer = new EvenThread(p, c);
Thread t1 = new Thread(producer, "Thread producer");
Thread t2 = new Thread(consumer, "Thread consumer");
t1.start();
t2.start();
}
}
class OddThread implements Runnable {
Semaphore p;
Semaphore c;
public OddThread(Semaphore p, Semaphore c) {
super();
this.p = p;
this.c = c;
}
int counter = 1;
#Override
public void run() {
while (true) {
try {
p.acquire(1);
System.out.println(Thread.currentThread().getName() + " "
+ counter);
if (counter == 10) {
break;
}
counter++;
c.release(1);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
class EvenThread implements Runnable {
Semaphore p;
Semaphore c;
int counter = 2;
public EvenThread(Semaphore p, Semaphore c) {
super();
this.p = p;
this.c = c;
}
#Override
public void run() {
while (true) {
try {
c.acquire(1);
System.out.println(Thread.currentThread().getName() + " "
+ counter);
if (counter == 10) {
break;
}
counter=counter+2;
p.acquire(1);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
Your code can't be correct, it's not even symmetrical.
p.acquire(1);
c.release(1);
c.acquire(1);
p.acquire(1);
Your EvenThread.run method acquires both c and p, rather than releasing one of them.
Note, however, that your code doesn't look like it'll exit properly even once it's been fixed. Consider: neither thread releases a semaphore before it exits, so one of the threads will inevitably be blocked.
here is your problem:
............
c.acquire(1);
System.out.println(Thread.currentThread().getName() + " "
+ counter);
if (counter == 10) {
break;
}
counter=counter+2;
p.acquire(1); <--deadlock this has already been acquired by the other thread.
..............
you should release p here, to allow the other thread to continue; so replace the line I indicated with p.release(1) and it should be fine.

JUnit test not executing all threads created within the test

I wrote a simple example for threading which generates tables for numbers starting from 1 to 20. when I tested it with main method it executes all the threads (prints all the messages), while all threads are not being run (all messages are not being printed) most of the times (sometimes it runs all the threads) when doing the same with JUnit test. I think there should not be any difference in terms of output.
Here is the class with main method:
public class Calculator implements Runnable {
private int number;
Calculator(final int number){
this.number = number;
}
#Override
public void run() {
for(int i = 1; i <= 10; i++){
System.out.printf("%s : %d * %d = %d \n", Thread.currentThread().getName(), number, i, number * i);
}
}
public static void main(String[] args){
Calculator calculator = null;
Thread thread = null;
for(int i = 1; i < 21; i ++){
calculator = new Calculator(i);
thread = new Thread(calculator);
System.out.println(thread.getName() + " Created");
thread.start();
System.out.println(thread.getName() + " Started");
}
}
}
When I invoke the main method it prints all the results.
Bellow is the code for JUnit test equivalent to the main method:
public class CalculatorTest {
private Calculator calculator;
private Thread thread;
#Test
public void testCalculator() {
for(int i = 1; i < 21; i ++){
calculator = new Calculator(i);
thread = new Thread(calculator);
System.out.println(thread.getName() + " Created");
thread.start();
System.out.println(thread.getName() + " Started");
}
}
}
When I run the above test case, the behavior of the output is not consistant in the scene that sometimes it prints all the messages and most of the times prints only a few and exits. Here is the output captured in case of the above JUnit test case:
Thread-0 Created
Thread-0 Started
Thread-1 Created
Thread-1 Started
Thread-2 Created
Thread-2 Started
Thread-3 Created
Thread-3 Started
Thread-4 Created
Thread-4 Started
Thread-5 Created
Thread-5 Started
Thread-6 Created
Thread-6 Started
Thread-7 Created
Thread-7 Started
Thread-8 Created
Thread-8 Started
Thread-9 Created
Thread-9 Started
Thread-10 Created
Thread-10 Started
Thread-11 Created
Thread-11 Started
Thread-12 Created
Thread-12 Started
Thread-13 Created
Thread-13 Started
Thread-14 Created
Thread-14 Started
Thread-15 Created
Thread-15 Started
Thread-16 Created
Thread-16 Started
Thread-17 Created
Thread-17 Started
Thread-18 Created
Thread-18 Started
Thread-19 Created
Thread-19 Started
Thread-0 : 1 * 1 = 1
Thread-0 : 1 * 2 = 2
Thread-0 : 1 * 3 = 3
Thread-0 : 1 * 4 = 4
Thread-0 : 1 * 5 = 5
Thread-0 : 1 * 6 = 6
Thread-0 : 1 * 7 = 7
Thread-0 : 1 * 8 = 8
Thread-0 : 1 * 9 = 9
Thread-0 : 1 * 10 = 10
Thread-2 : 3 * 1 = 3
Thread-2 : 3 * 2 = 6
Thread-2 : 3 * 3 = 9
Thread-2 : 3 * 4 = 12
Thread-2 : 3 * 5 = 15
Thread-2 : 3 * 6 = 18
Thread-2 : 3 * 7 = 21
Output ends here without printing the remaining messages in other threads/ executing other threads.
Can somebody help me to understand the reason behind this. Thanks in advance.
JUnit is exiting the test method early. You need to wait for all of the threads to complete before you exit the testCalculator() method.
An easy way to do that is by using a CountDownLatch.
Initialize a CountDownLatch with CountDownLatch latch = new CountDownLatch(20).
Pass each Calculator runnable a reference to the latch. At the end of the run() method, call latch.countDown().
At the end of the testCalculator() method call latch.await(). This will block until latch.countDown() has been called 20 times (i.e. when all threads have completed).
Your test method finishes before all the spawned threads are finished. When the JUnit executor finishes, all spawned threads are killed.
If you want to run this kind of test, you should keep a collection of the threads you have created and join() each of them at the end of your test method. The calls to join() each thread are executed in a second loop (following the loop that starts all the threads).
Something like this:
#Test
public void testCalculator() {
List<Thread> threads = new ArrayList<>();
for (int i = 1; i < 21; i++) {
calculator = new Calculator(i);
thread = new Thread(calculator);
threads.add(thread);
System.out.println(thread.getName() + " Created");
thread.start();
System.out.println(thread.getName() + " Started");
}
for (Thread thread : threads) {
thread.join();
}
}
If you want to have the threads all start around the same time (e.g., if your loop that is creating the threads does some non-trivial work each time through the loop):
#Test
public void testCalculator() {
List<Thread> threads = new ArrayList<>();
for (int i = 1; i < 21; i++) {
threads.add(new Thread(new Calculator(i)));
}
for (Thread thread : threads) {
thread.start();
}
for (Thread thread : threads) {
thread.join();
}
}

Resources