JavaFX.Scene.Robot crashes / UI freezes - multithreading

Question

I am trying to write a simple JavaFX app which acts as an auto clicker for a game I play. I choose two Points that the mouse should click alternately. Everything works fine until the Robot needs to do his work.
When I put it like this:
robot.mouseMove(join);
Thread.sleep(2000);
robot.mouseClick(MouseButton.PRIMARY);
Thread.sleep(2000);
robot.mouseMove(accept);
Thread.sleep(2000);
robot.mouseClick(MouseButton.PRIMARY);
Thread.sleep(2000);
my App crashes. I've read things up online and it seems like you should not sleep in the JavaFX application thread. My new approach was to create a new thread that takes care of the clicking from the application thread like this:
clicker = new Clicker(join, accept);
Thread clickerThread = new Thread(clicker);
clickerThread.start();
And here how it looks in Clicker:
public void run() {
while (running){
try {
robot.mouseMove(join);
Thread.sleep(2000);
robot.mouseClick(MouseButton.PRIMARY);
Thread.sleep(2000);
robot.mouseMove(accept);
Thread.sleep(2000);
robot.mouseClick(MouseButton.PRIMARY);
Thread.sleep(2000);
} catch (InterruptedException e) {
System.out.println("Clicker sleep interrupted!");
}
}
}
However with the new approach I suddenly get this error:
Exception in thread "Thread-3" java.lang.IllegalStateException: This operation is permitted on the event thread only; currentThread = Thread-3
Does anyone know how I could fix this problem?

Answer 1

When you want to execute a periodic foreground task on the JavaFX Application Thread you should first consider using an animation. Here's an example of using a Timeline:
Point2D join = ...;
Point2D accept = ...;
Robot robot = ...;
Timeline timeline = new Timeline(
new KeyFrame(Duration.ZERO, e -> robot.mouseMove(join)),
new KeyFrame(Duration.seconds(2), e -> robot.mouseClick(MouseButton.PRIMARY)),
new KeyFrame(Duration.seconds(4), e -> robot.mouseMove(accept)),
new KeyFrame(Duration.seconds(6), e -> robot.mouseClick(MouseButton.PRIMARY)),
new KeyFrame(Duration.seconds(8))
);
timeline.play();
The above will execute each KeyFrame's on-finished handler two seconds after the previous one (the first one immediately after the animation is started). The last KeyFrame matches your final call to sleep though it may not be necessary. All this will occur on the JavaFX Application Thread.
You can configure an animation to replay a certain number of times, or even forever, by setting its cycleCount property. For example:
timeline.setCycleCount(5); // play 5 times then stop
// or
timeline.setCycleCount(Animation.INDEFINITE); // play forever

Answer 2

You need to execute the mouseMove on the UI thread try wrapping the calls with Platform.runlater(() -> robot.mouseMove(MouseButton.PRIMARY));
public void run() {
while (running){
try {
Platform.runlater(() -> robot.mouseMove(join));
Thread.sleep(2000);
Platform.runlater(() -> robot. mouseClick(MouseButton.PRIMARY));
Thread.sleep(2000);
Platform.runlater(() -> robot.mouseMove(accept));
Thread.sleep(2000);
} catch (InterruptedException e) {
System.out.println("Clicker sleep interrupted!");
}
}
}
Platform.runLater() will schedule an action on the main thread and return immediately. It may take some time before the action is actually executed, in small applications this delay is usually not perceptible. If the delay were long it may happen that you schedule your action, sleep and schedule again before the first action was executed. If you want your clickerThread to wait for the action to execute before continuing then you will need some form of synchronisation, see below for an example of a method that wraps Platform.runlater() with a lock.
public void waitUntilExecutedOnMainThread(Runnable runnable){
Semaphore semaphore = new Semaphore(0);
Platform.runLater(() -> {
try {
runnable.run();
} finally {
semaphore.release();
}
});
try {
semaphore.acquire();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
waitUntilExecutedOnMainThread(() -> robot. mouseClick(MouseButton.PRIMARY))