How can I call the akka scheduler from a thread outside of the Akka system?
I have an actor that will instantiate a few java classes, which have callbacks that are run on their own threads. I'd like to schedule execution from these callbacks:
pseudocode:
class GPIOActor extends Actor with PubSubActor {
class DebouncerListener(pin: Pin) extends GpioPinListenerDigital {
override def stateChangeEvent(event: GpioPinDigitalStateChangeEvent) {
if (notAlreadyWaitingForBounceToFinish)
context.system.scheduler.scheduleOnce(debouncePollPeriod, new Runnable() {
def run(): Unit = {
if (stillBouncing) {
context.system.scheduler.scheduleOnce(debouncePollPeriod, this)(context.dispatcher)
} else self ! PinStateChangeEvent(pin.getValue)
}
}
})(context.dispatcher)
}
val myPin = GPIOPinListener(RapiPin)
myPin.registerCallback(new DebouncerListener(myPin))
(...)
def receive: Receive = {
case PinStateChangeEvent(newValue) => notifySubscribers(newValue)
}
}
Related
I'm new with kotlin I'm trying to run several requests to a web in parallel threads
so far I got
class HttpClient {
private val DEFAULT_BASE_URL = "https://someapi"
fun fetch(endPoint: String, page: Int): String {
FuelManager.instance.basePath = DEFAULT_BASE_URL
val (_, response, _) = endPoint.httpGet(listOf("page" to page)).response()
return String(response.data)
}
fun headers(endPoint: String): Headers {
FuelManager.instance.basePath = DEFAULT_BASE_URL
val (_, response, _) = endPoint.httpGet(listOf("page" to 1)).response()
return response.headers
}
}
and the class that runs the whole process
class Fetcher(private val page: Int) {
suspend fun run(): String = coroutineScope {
async {
HttpClient().fetch(DEFAULT_ENDPOINT, page)
}
}.await()
companion object {
private const val DEFAULT_ENDPOINT = "endpoint"
suspend fun fetchAll(): MutableList<String> {
val totalThreads = (totalCount() / pageSize()) + 1
return runBlocking {
var deck: MutableList<String> = mutableListOf()
for (i in 1..totalThreads) {
deck.add(Fetcher(i).run())
}
deck
}
}
private fun pageSize(): Int {
return HttpClient().headers(DEFAULT_ENDPOINT)["page-size"].first().toInt()
}
private fun totalCount(): Int {
return HttpClient().headers(DEFAULT_ENDPOINT)["total-count"].first().toInt()
}
}
}
I'm looking to mirror the Thread.join() from Java. Could you give me some pointers on how to improve my code to achieve that?
Also if not much asking, could you suggest a book/example set on this subject?
Thanks for your help in advance!
A few points:
If you're going to be using coroutines in a project, you'll mostly want to be exposing suspending functions instead of blocking functions. I don't use Fuel, but I see it has a coroutines library with suspend function versions of its blocking functions. Usually, suspend functions that unwrap an asynchronous result have the word "await" in them. I don't know for sure what response() is since I don't use fuel, but if I had to guess, you can use awaitResponse() instead and then make the functions suspend functions.
Not related to coroutines, but there's almost no reason to ever use the String constructor to wrap another String, since Strings are immutable. (The only reason you would ever need to copy a String in memory like that is maybe if you were using it in some kind of weird collection that uses identity comparison instead of `==`` comparison, and you need it to be treated as a different value.)
Also not related to coroutines, but HttpClient in your case should be a singleton object since it holds no state. Then you won't need to instantiate it when you use it or worry about holding a reference to one in a property.
Never use runBlocking in a suspend function. A suspend function must never block. runBlocking creates a blocking function. The only two places runBlocking should ever appear in an application are at the top level main function of a CLI app, or in an app that has both coroutines and some other thread-management library and you need to convert suspend functions into blocking non-suspend functions so they can be used by the non-coroutine-based code.
There's no reason to immediately follow async() with await() if you aren't doing it in parallel with something else. You could just use withContext instead. If you don't need to use a specific dispatcher to call the code, which you don't if it's a suspend function, then you don't even need withContext. You can just call suspend functions directly in your coroutine.
There's no reason to use coroutineScope { } to wrap a single child coroutine. It's for running multiple child coroutines and waiting for all of them.
So, if we change HttpClient's functions into suspend functions, then Fetcher.run becomes very simple.
I also think that it's kind of weird that Fetcher is a class with a single property that is only used in a one-off fashion with its only function. Instead, it would be more straight-forward for Fetcher to be a singleton object and for run to have the parameter it needs. Then you won't need a companion object either since Fetcher as an object can directly host those functions.
Finally, the part you were actually asking about: to run parallel tasks in a coroutine, use coroutineScope { } and then launch async coroutines inside it and await them. The map function is handy for doing this with something you can iterate, and then you can use awaitAll(). You can also get totalCount and pageSize in parallel.
Bringing that all together:
object HttpClient {
private val DEFAULT_BASE_URL = "https://someapi"
suspend fun fetch(endPoint: String, page: Int): String {
FuelManager.instance.basePath = DEFAULT_BASE_URL
val (_, response, _) = endPoint.httpGet(listOf("page" to page)).awaitResponse()
return response.data
}
suspend fun headers(endPoint: String): Headers {
FuelManager.instance.basePath = DEFAULT_BASE_URL
val (_, response, _) = endPoint.httpGet(listOf("page" to 1)).awaitResponse()
return response.headers
}
}
object Fetcher() {
suspend fun run(page: Int): String =
HttpClient.fetch(DEFAULT_ENDPOINT, page)
private const val DEFAULT_ENDPOINT = "endpoint"
suspend fun fetchAll(): List<String> {
val totalThreads = coroutineScope {
val totalCount = async { totalCount() }
val pageSize = async { pageSize() }
(totalCount.await() / pageSize.await()) + 1
}
return coroutineScope {
(1..totalThreads).map { i ->
async { run(i) }
}.awaitAll()
}
}
private suspend fun pageSize(): Int {
return HttpClient.headers(DEFAULT_ENDPOINT)["page-size"].first().toInt()
}
private suspend fun totalCount(): Int {
return HttpClient.headers(DEFAULT_ENDPOINT)["total-count"].first().toInt()
}
}
I changed MutableList to List, since it's simpler, and usually you don't need a MutableList. If you really need one you can call toMutableList() on it.
I would like to get your feedback on the code below.
I'm wondering if it's safe to call currentSession.onNext(result.session)
from inside the SessionManager.signIn stream.
My first intuition is to say NO because of multithreading and synchronization issues, meaning, based on this code, I could be calling currentSession.onNext(result.session) from different threads.
Here is the code, please let me know what you think! Thanks
SessionManager which is a singleton
#Singleton
class SessionManager #Inject constructor(
private val sessionService: SessionService,
){
val currentSession = BehaviorProcessor.create<Session>()
fun signIn(login: String, password: String): Single<Boolean> =
sessionService.signIn(login, password)
.doOnNext(result ->
if (session is Success) {
currentSession.onNext(result.session)
}
).map { result ->
when (result) {
is Success -> true
else -> false
}
}
.subscribeOn(Schedulers.io())
}
HomeView which is a random View subscribing to the SessionManager's signIn stream
class HomeView(val context: Context) : View(context) {
#Inject
lateinit var sessionManager: SessionManager
private val disposables = CompositeDisposable()
override fun onAttachedToWindow() {
super.onAttachedToWindow()
disposables.add(sessionManager.signIn("username", "password")
.distinctUntilChanged()
.observeOn(AndroidSchedulers.mainThread())
.subscribe { result ->
textView.text = if (result) "Success" else "Fail"
})
}
override fun onDetachedFromWindow() {
super.onDetachedFromWindow()
disposables.clear()
}
}
A random View observing the currentSession from SessionManager
class RandomView(val context: Context) : View(context) {
#Inject
lateinit var sessionManager: SessionManager
private val disposables = CompositeDisposable()
override fun onAttachedToWindow() {
super.onAttachedToWindow()
disposables.add(sessionManager.currentSession
.distinctUntilChanged()
.observeOn(AndroidSchedulers.mainThread())
.subscribe { session -> userTextView.text = session.userName })
}
override fun onDetachedFromWindow() {
super.onDetachedFromWindow()
disposables.clear()
}
}
The documentation of BehaviorProcessor says:
Calling onNext(Object), offer(Object), onError(Throwable) and onComplete() is required to be serialized (called from the same thread or called non-overlappingly from different threads through external means of serialization). The FlowableProcessor.toSerialized() method available to all FlowableProcessors provides such serialization and also protects against reentrance (i.e., when a downstream Subscriber consuming this processor also wants to call onNext(Object) on this processor recursively).
So if you define it like this:
val currentSession = BehaviorProcessor.create<Session>().toSerialized()
then you can safely call onNext from any thread, it will not cause any synchronisation problems.
Notes:
I agree that the update of the processor should be in a doOnNext instead of the map.
I think it would be better to use a Completable instead of a Single<Boolean>, and use Rx errors to indicate what prevented signing in. You should also define the error handlers in the subscribe methods.
Basically all I want to do is getting all users from my database, which worked fine until the very moment i wanted to use deadbolt for it:
I think the 4 Threads(number of processors) of the fork-join-executor are already all used and then there is somekind of deadlock.
Things I tried:
Raise the number of threads the executor has, so however play/akka ignores my settings
Define another execution context for the futures in the controller, but this does not prevent deadlocks since more than four threads still wait at each other
use a thread-pool-executor, but my settings are ignored
A mixed scala/java code from here:
class UserController {
def getUserList = deadbolt.Restrict(List(Array("Admin")))(){ implicit request =>
Future {
val users = userModel.list
val json = Json.toJson(users)
Ok(json.toString)
}(
}
}
The User Model is essentially nothing more than:
public class UserModel {
private MongoClient client = new MongoClient();
private Morphia morphia = new Morphia();
protected Datastore datastore = morphia.createDatastore(client, "timetracking");
public List<User> list(){
return datastore.find(User.class).asList();
}
public User findUserByName(String name){
User found = datastore.createQuery(User.class).field("username").equal(name).get();
return found;
}
}
Authorization Handler:
class AuthorizationHandler extends DeadboltHandler {
val model = new UserModel
override def getSubject[A](request: AuthenticatedRequest[A]): Future[Option[Subject]] =
Future {
blocking {
request.subject match {
case Some(user) =>
request.subject
case None =>
val username = request.session.get("username")
if (username.isDefined) {
val user = model.findUserByName(username.get)
if (user == null) {
None
} else {
val subject = new ScalaSubject(user.getUsername, user.getRole)
Some(subject)
}
} else {
None
}
}
}
}
Defining a seperate deadbolt context does not help:
package deadbolt.scala
import be.objectify.deadbolt.scala.DeadboltExecutionContextProvider
import be.objectify.deadbolt.scala.cache.HandlerCache
import play.api.inject.{Binding, Module}
import play.api.{Configuration, Environment}
class DeadBoldModule extends Module {
override def bindings(environment: Environment,
configuration: Configuration): Seq[Binding[_]] = Seq(
bind[HandlerCache].to[TimeTrackerHandelCache],
bind[DeadboltExecutionContextProvider].to[ThreadPoolProvider]
)
}
Custom context provider:
package deadbolt.scala
import java.io.InvalidObjectException
import java.util.concurrent.Executors
import be.objectify.deadbolt.scala.DeadboltExecutionContextProvider
import scala.concurrent.ExecutionContext
class ThreadPoolProvider extends DeadboltExecutionContextProvider {
override def get(): ExecutionContext = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(100))
}
When I try this, throwing some random exception, it is never thrown:
package deadbolt.scala
import java.io.InvalidObjectException
import java.util.concurrent.Executors
import be.objectify.deadbolt.scala.DeadboltExecutionContextProvider
import scala.concurrent.ExecutionContext
class ThreadPoolProvider extends DeadboltExecutionContextProvider {
override def get(): ExecutionContext = throw new IllegalAccessError("asd");ExecutionContext.fromExecutor(Executors.newFixedThreadPool(100))
}
It was not deadbolts fault, but however the MongoClient opened a new thread when it was instantiated.Which happened in our project quite often, but was not closed properly thereby blocking the threadpool. We used a Singleton and everything worked fine.
How large is Scala's thread pool for futures?
My Scala application makes many millions of future {}s and I wonder if there is anything I can do to optimize them by configuring a thread pool.
Thank you.
This answer is from monkjack, a comment from the accepted answer. However, one can miss this great answer so I'm reposting it here.
implicit val ec = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(10))
If you just need to change the thread pool count, just use the global executor and pass the following system properties.
-Dscala.concurrent.context.numThreads=8 -Dscala.concurrent.context.maxThreads=8
You can specify your own ExecutionContext that your futures will run in, instead of importing the global implicit ExecutionContext.
import java.util.concurrent.Executors
import scala.concurrent._
implicit val ec = new ExecutionContext {
val threadPool = Executors.newFixedThreadPool(1000)
def execute(runnable: Runnable) {
threadPool.submit(runnable)
}
def reportFailure(t: Throwable) {}
}
best way to specify threadpool in scala futures:
implicit val ec = new ExecutionContext {
val threadPool = Executors.newFixedThreadPool(conf.getInt("5"));
override def reportFailure(cause: Throwable): Unit = {};
override def execute(runnable: Runnable): Unit = threadPool.submit(runnable);
def shutdown() = threadPool.shutdown();
}
class ThreadPoolExecutionContext(val executionContext: ExecutionContext)
object ThreadPoolExecutionContext {
val executionContextProvider: ThreadPoolExecutionContext = {
try {
val executionContextExecutor: ExecutionContextExecutor = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(25))
new ThreadPoolExecutionContext(executionContextExecutor)
} catch {
case exception: Exception => {
Log.error("Failed to create thread pool", exception)
throw exception
}
}
}
}
I have the following Groovy code:
abstract class Actor extends Script {
synchronized void proceed() {
this.notify()
}
synchronized void pause() {
wait()
}
}
class MyActor extends Actor {
def run() {
println "hi"
pause()
println "hi again"
}
}
def theactor = new MyActor()
theactor.run()
theactor.proceed()
When I run the code, I want the code to output "hi" and "hi again". Instead, it just stops at "hi" and gets stuck on the pause() function. Any idea on how I could continue the program?
As Brian says, multithreading and concurrency is a huge area, and it is easier to get it wrong, than it is to get it right...
To get your code working, you'd need to have something like this:
abstract class Actor implements Runnable {
synchronized void proceed() { notify() }
synchronized void pause() { wait() }
}
class MyActor extends Actor {
void run() {
println "hi"
pause()
println "hi again"
}
}
def theactor = new MyActor() // Create an instance of MyActor
def actorThread = new Thread( theactor ) // Create a Thread to run this instance in
actorThread.start() // Thread.start() will call MyActor.run()
Thread.sleep( 500 ) // Make the main thread go to sleep for some time so we know the actor class is waiting
theactor.proceed() // Then call proceed on the actor
actorThread.join() // Wait for the thread containing theactor to terminate
However, if you are using Groovy, I would seriously consider using a framework like Gpars which brings concurrency to Groovy and is written by people who really know their stuff. I can't think of anything that llows this sort of arbitrary pausing of code however... Maybe you could design your code to fit one of their usage patterns instead?
Threading is a big topic and there are libraries in Java to do many common things without working with the Thread API directly. One simple example for 'Fire and Forget' is Timer.
But to answer your immediate question; another thread needs to notify your thread to continue. See the docs on wait()
Causes current thread to wait until another thread invokes the
notify() method or the notifyAll() method for this object. In other
words, this method behaves exactly as if it simply performs the call
wait(0).
One simple 'fix' is to just add a fixed duration to your wait call just to continue with your exploration. I would suggest the book 'Java Concurrency in Practice'.
synchronized void pause() {
//wait 5 seconds before resuming.
wait(5000)
}