Kotlin: How to convert test which uses Thread.sleep to RxJava TestScheduler - multithreading

I am writing an instrumental test, it checks if when i cache something into a Rx buffer and after some interval(10 seconds) this Subject insert buffered values to my Room database.
The test is correct when I use Thread.sleep(syncTimeInterval). I want to write this same test using TestScheduler.
Here it is with Thread.sleep version(which pass the test):
#Test
fun testMultipleLogs() {
val loadAllCloudCallBefore = appDatabase.logCloudCallDao().loadAll()
val loadAllLogNewSessionBefore = appDatabase.logNewSessionDao().loadAll()
assertEquals(0, loadAllCloudCallBefore.size)
assertEquals(0, loadAllLogNewSessionBefore.size)
Observable.interval(1, TimeUnit.SECONDS)
.take(20)
.subscribe { logManager.logNewSession() }
Observable.interval(1, TimeUnit.SECONDS)
.take(20)
.subscribe { logManager.logCloudCall("url", "callgoup") }
Observable.interval(1, TimeUnit.SECONDS)
.take(20)
.subscribe { logManager.logNewSession() }
Observable.interval(1, TimeUnit.SECONDS)
.take(20)
.subscribe { logManager.logCloudCall("url", "callgoup") }
Observable.interval(1, TimeUnit.SECONDS)
.take(20)
.subscribe { logManager.logNewSession() }
Observable.interval(1, TimeUnit.SECONDS)
.take(20)
.subscribe { logManager.logCloudCall("url", "callgoup") }
Thread.sleep(30000)
val loadAllCloudCallAfter = appDatabase.logCloudCallDao().loadAll()
val loadAllLogNewSessionAfter = appDatabase.logNewSessionDao().loadAll()
assertEquals(60, loadAllCloudCallAfter.size)
assertEquals(60, loadAllLogNewSessionAfter.size)
}
And here, this test not passes, the size after expecting time advanced by TestScheduler is 0(not 60)
#Test
fun testMultipleLogs() {
var testScheduler: TestScheduler = TestScheduler()
val loadAllCloudCallBefore = appDatabase.logCloudCallDao().loadAll()
val loadAllLogNewSessionBefore = appDatabase.logNewSessionDao().loadAll()
assertEquals(0, loadAllCloudCallBefore.size)
assertEquals(0, loadAllLogNewSessionBefore.size)
Observable.interval(1, TimeUnit.SECONDS, testScheduler)
.take(20)
.subscribe { logManager.logNewSession() }
Observable.interval(1, TimeUnit.SECONDS, testScheduler)
.take(20)
.subscribe { logManager.logCloudCall("url", "callgoup") }
Observable.interval(1, TimeUnit.SECONDS, testScheduler)
.take(20)
.subscribe { logManager.logNewSession() }
Observable.interval(1, TimeUnit.SECONDS, testScheduler)
.take(20)
.subscribe { logManager.logCloudCall("url", "callgoup") }
Observable.interval(1, TimeUnit.SECONDS, testScheduler)
.take(20)
.subscribe { logManager.logNewSession() }
Observable.interval(1, TimeUnit.SECONDS, testScheduler)
.take(20)
.subscribe { logManager.logCloudCall("url", "callgoup") }
testScheduler.advanceTimeBy(21, TimeUnit.SECONDS)
val loadAllCloudCallAfter = appDatabase.logCloudCallDao().loadAll()
val loadAllLogNewSessionAfter = appDatabase.logNewSessionDao().loadAll()
assertEquals(60, loadAllCloudCallAfter.size)
assertEquals(60, loadAllLogNewSessionAfter.size)
}
How can i test this case properly? Is there a way?
UPDATE
The functions in LogManager looks like this:
fun logCloudCall(url: String, callGroup: String) {
val logCloudCall = LogCloudCall(url = url, callGroup = callGroup, date = Converter.GENERAL_DATE_FORMAT.format(Date()))
Log.v("LogManager", logCloudCall.toString())
addLog(logCloudCall)
}
fun logNewSession() {
val logNewSession =
LogNewSession(
date = Converter.GENERAL_DATE_FORMAT.format(Date()))
Log.v("LogManager", logNewSession.toString())
addLog(logNewSession)
}
fun addLog(logEvent: LogEvent) {
source.onNext(logEvent)
}
And this is the mechanism which i use in my LogManager init:
val source = PublishSubject.create<LogEvent>().toSerialized()
var logRepository: LogRepository
init {
logRepository = LogRepositoryImpl(context)
configureSubject()
}
fun configureSubject() {
source
.buffer(10, TimeUnit.SECONDS)
.subscribe { bufferedData -> proceedValues(bufferedData) }
}

The following test passes:
#Test
fun foo() {
val testScheduler = TestScheduler()
var count = 0
Observable.interval(1, TimeUnit.SECONDS, testScheduler)
.take(20)
.subscribe { count++ }
testScheduler.advanceTimeBy(21, SECONDS)
assert(count == 20)
}
That is, your test code looks correct, but the result is not correct. The only unknown here is the logManager code. Is there any threading in that class? That may explain why the count is still 0: you may have a race condition.
It's probably due to the buffer call. buffer internally uses the a computation Scheduler:
public final Observable<List<T>> buffer(long timespan, TimeUnit unit) {
return buffer(timespan, unit, Schedulers.computation(), Integer.MAX_VALUE);
}
This will probably result in the threading issue you're seeing.

Related

How can an App write and read file in Documents folder?

I wrote an App, in Kotlin with Android Studio that write some strings to a file.
All work, I can write and read inside the App, but I can't see the file looking in Documents folder.
How can I use the folder Documents as a storage space?
Thank you
These are the function I use:
fun saveTextFile(view: View, nomeFile: String, testo: String, contesto: Context){
var fileStream: FileOutputStream
try {
fileStream = contesto.openFileOutput(nomeFile, MODE_APPEND) // OK esegue append
fileStream.write(testo.toByteArray())
fileStream.close()
}catch (e: Exception){
e.printStackTrace()
}
}
fun readTextFile(view: View, nomeFile: String, contesto: Context): String{
var fileInputStream: FileInputStream? = null
fileInputStream = contesto.openFileInput(nomeFile)
var inputStreamReader: InputStreamReader = InputStreamReader(fileInputStream)
val bufferedReader: BufferedReader = BufferedReader(inputStreamReader)
val stringBuilder: StringBuilder = StringBuilder()
var text: String? = null
while ({ text = bufferedReader.readLine(); text }() != null) {
stringBuilder.append(text)
}
inputStreamReader.close();
return(stringBuilder.toString())
}
Thank you, Livio
For writing in Documents folder of your device , you just need to make use of MediaStore for the same. You can take input for this function anything that you want like String , bitmap , PdfDocument and other's too .
For Your UseCase you can do the following ,
Global Variable :
private var imageUri: Uri? = null
override suspend fun saveDocument(context : Context, text : String) {
withContext(Dispatchers.IO) {
try {
val collection =
MediaStore.Files.getContentUri(MediaStore.VOLUME_EXTERNAL_PRIMARY)
val dirDest = File(
Environment.DIRECTORY_DOCUMENTS,
context.getString(R.string.app_name)
)
val date = System.currentTimeMillis()
val fileName = "$date.txt"
val contentValues = ContentValues().apply {
put(MediaStore.MediaColumns.DISPLAY_NAME, fileName)
put(MediaStore.MediaColumns.RELATIVE_PATH,
"$dirDest${File.separator}")
put(MediaStore.Files.FileColumns.IS_PENDING, 1)
}
}
val imageUri = context.contentResolver.insert(collection, contentValues)
withContext(Dispatchers.IO) {
imageUri?.let { uri ->
context.contentResolver.openOutputStream(uri, "w").use { out -> out?.write(text.toByteArray())
}
contentValues.clear()
contentValues.put(MediaStore.Files.FileColumns.IS_PENDING, 0)
context.contentResolver.update(uri, contentValues, null, null)
}
}
} catch (e: FileNotFoundException) {
null
}
}
}
For Updating the already existing file , do the following . After creating file for the first time I have saved the imageUri in a global variable (If you want to store it permanently / or for a while you can use Jetpack Datastore / Shared Preference to save the same ):
suspend fun updateData(context: Context,text : String){
withContext(Dispatchers.IO) {
try {
val collection =
MediaStore.Files.getContentUri(MediaStore.VOLUME_EXTERNAL_PRIMARY)
val dirDest = File(
Environment.DIRECTORY_DOCUMENTS,
context.getString(R.string.app_name)
)
val fileName = "test.txt"
val contentValues = ContentValues().apply {
put(MediaStore.MediaColumns.DISPLAY_NAME, fileName)
put(
MediaStore.MediaColumns.RELATIVE_PATH,
"$dirDest${File.separator}"
)
put(MediaStore.Files.FileColumns.IS_PENDING, 1)
}
withContext(Dispatchers.IO) {
imageUri?.let { uri ->
context.contentResolver.openOutputStream(uri, "wa").use { out ->
out?.write(text.toByteArray())
}
contentValues.clear()
contentValues.put(MediaStore.Files.FileColumns.IS_PENDING, 0)
context.contentResolver.update(uri, contentValues, null, null)
}
}
} catch (e: FileNotFoundException) {
null
}
}
}
For Reading the File , Do the following :
suspend fun read(context: Context, source: Uri): String = withContext(Dispatchers.IO) {
val resolver: ContentResolver = context.contentResolver
resolver.openInputStream(source)?.use { stream -> stream.readText() }
?: throw IllegalStateException("could not open $source")
}
private fun InputStream.readText(charset: Charset = Charsets.UTF_8): String =
readBytes().toString(charset)
This is how the final code looks like :
class MainActivity : AppCompatActivity() {
private lateinit var btn: Button
private var imageUri: Uri? = null
private lateinit var btn2: Button
private lateinit var btn3 : Button
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
btn = findViewById(R.id.btnAdd)
btn2 = findViewById(R.id.getText)
btn3 = findViewById(R.id.updateText)
btn.setOnClickListener {
lifecycleScope.launch {
saveDocument(applicationContext, "Original ")
}
}
btn3.setOnClickListener {
lifecycleScope.launch {
updateData(applicationContext,"Appended")
}
}
btn2.setOnClickListener {
lifecycleScope.launch {
imageUri?.let { it1 ->
val data = read(applicationContext, it1)
Toast.makeText(applicationContext, "The data is $data ", Toast.LENGTH_LONG)
.show()
}
}
}
}
suspend fun saveDocument(context: Context, text: String) {
withContext(Dispatchers.IO) {
try {
val collection =
MediaStore.Files.getContentUri(MediaStore.VOLUME_EXTERNAL_PRIMARY)
val dirDest = File(
Environment.DIRECTORY_DOCUMENTS,
context.getString(R.string.app_name)
)
val date = System.currentTimeMillis()
val fileName = "test.txt"
val contentValues = ContentValues().apply {
put(MediaStore.MediaColumns.DISPLAY_NAME, fileName)
put(
MediaStore.MediaColumns.RELATIVE_PATH,
"$dirDest${File.separator}"
)
put(MediaStore.Files.FileColumns.IS_PENDING, 1)
}
imageUri = context.contentResolver.insert(collection, contentValues)
withContext(Dispatchers.IO) {
imageUri?.let { uri ->
context.contentResolver.openOutputStream(uri, "w").use { out ->
out?.write(text.toByteArray())
}
contentValues.clear()
contentValues.put(MediaStore.Files.FileColumns.IS_PENDING, 0)
context.contentResolver.update(uri, contentValues, null, null)
}
}
} catch (e: FileNotFoundException) {
null
}
}
}
suspend fun updateData(context: Context, text: String) {
withContext(Dispatchers.IO) {
try {
val collection =
MediaStore.Files.getContentUri(MediaStore.VOLUME_EXTERNAL_PRIMARY)
val dirDest = File(
Environment.DIRECTORY_DOCUMENTS,
context.getString(R.string.app_name)
)
val fileName = "test.txt"
val contentValues = ContentValues().apply {
put(MediaStore.MediaColumns.DISPLAY_NAME, fileName)
put(
MediaStore.MediaColumns.RELATIVE_PATH,
"$dirDest${File.separator}"
)
put(MediaStore.Files.FileColumns.IS_PENDING, 1)
}
withContext(Dispatchers.IO) {
imageUri?.let { uri ->
context.contentResolver.openOutputStream(uri, "wa").use { out ->
out?.write(text.toByteArray())
}
contentValues.clear()
contentValues.put(MediaStore.Files.FileColumns.IS_PENDING, 0)
context.contentResolver.update(uri, contentValues, null, null)
}
}
} catch (e: FileNotFoundException) {
null
}
}
}
suspend fun read(context: Context, source: Uri): String = withContext(Dispatchers.IO) {
val resolver: ContentResolver = context.contentResolver
resolver.openInputStream(source)?.use { stream -> stream.readText() }
?: throw IllegalStateException("could not open $source")
}
private fun InputStream.readText(charset: Charset = Charsets.UTF_8): String =
readBytes().toString(charset)
I have three buttons . With the first I create a file , then the uri gets stored in the global variable . Then onClick of second button I add to the already existing file and then read the file using the third button using the same imageUri stored in the global variable
This is the demo for the same . Check when the buttons are being pressed and the output in the form of Toast at the bottom .

How get an JSONObject from a JSONArray?

I have a problem when try get the json object from json array. The catched error say that JSONArray cannot be convertered to JSONObject. Effectively can get JSON response from web service, but i don't know how convert array to object
error message
org.json.JSONException: Value [{"codaspirante":"1","nombre":"Saul","apellido":"Goodman","mail":"sg#g.com","telefono":"012034948123","codusuario":"0"},{"codaspirante":"2","nombre":"Lalo","apellido":"Salamanca","mail":"ls#g.com","telefono":"12351233","codusuario":"10"},{"codaspirante":"3","nombre":"Walter","apellido":"White","mail":"ww#g.com","telefono":"54843439","codusuario":"10"},{"codaspirante":"4","nombre":"Gustavo","apellido":"Frings","mail":"gf#g.com","telefono":"845738848434","codusuario":"10"}] at 0 of type org.json.JSONArray cannot be converted to JSONObject
Method:
private void buscarCandidatos_Serivicio(String URL){
JsonArrayRequest jsonArrayRequest = new JsonArrayRequest(Request.Method.POST, URL, null, new Response.Listener<JSONArray>() {
#Override
public void onResponse(JSONArray response) {
try {
for (int i = 0; i < response.length(); i++) {
JSONObject candidato = response.getJSONObject(i);
Candidato c = new Candidato(
candidato.getInt("codaspirante"),
candidato.getString("nombre"),
candidato.getString("apellido"),
candidato.getString("mail"),
candidato.getString("telefono"),
candidato.getInt("codusuario")
);
listaCandidatos.add(c);
}
} catch (JSONException e) {
Toast.makeText(getApplicationContext(), e.getMessage().toString(), Toast.LENGTH_SHORT).show();
}
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
Toast.makeText(getApplicationContext(), error.getMessage(), Toast.LENGTH_SHORT).show();
}
}
);
RequestQueue rq= Volley.newRequestQueue(this);
rq.add(jsonArrayRequest);
}
The error is because response.getJSONObject(0) is returning a JSONArray and not a JSONObject.
I am guessing you want to fetch the data from that Array.
In your On response Method you should add response = response.getJSONArray(0).
public void onResponse(JSONArray response) {
try {
response = response.getJSONArray(0);
for (int i = 0; i < response.length(); i++) {
That should fix your problem.
val stringRequest = object : StringRequest(
Request.Method.POST, Config."your URL",
Response.Listener<String> { response ->
try {
val obj = JSONObject(response)
if (!obj.getBoolean("error")) {
val userssListArray = obj.getJSONArray("arrayname")
for (i in 0 until userssListArray.length()) {
var usersObj = userssListArray.getJSONObject(i)
}
} else {
if (obj.getString("message") == "Your login expired please re login") {
val intent = Intent(this, LoginActivity::class.java)
startActivity(intent)
finish()
}
}
} catch (e: JSONException) {
e.printStackTrace()
}
},
Response.ErrorListener {
Toast.makeText(this, "Network Error Try Again...", Toast.LENGTH_LONG).show()
}) {
#Throws(AuthFailureError::class)
override fun getParams(): Map<String, String> {
val params = HashMap<String, String>()
return params
}
}
stringRequest.retryPolicy = DefaultRetryPolicy(
15000,
DefaultRetryPolicy.DEFAULT_MAX_RETRIES,
DefaultRetryPolicy.DEFAULT_BACKOFF_MULT)
//adding request to queue
VolleySingleton.instance?.addToRequestQueue(stringRequest)
you can use volley like this

Spark streaming SQS with checkpoint enable

I have went through multiple sites like
https://spark.apache.org/docs/latest/streaming-programming-guide.html
https://data-flair.training/blogs/spark-streaming-checkpoint/
https://docs.databricks.com/spark/latest/rdd-streaming/developing-streaming-applications.html
Some links talk about the how we can code but it's so abstract that I needed a lot of time to figure out how this actually works
After a long fight I am able to setup the streaming code with checkpoint, adding here to help others
import java.util.concurrent.Executors
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain
import com.amazonaws.regions.Regions
import com.amazonaws.services.sqs.model.Message
import com.fasterxml.jackson.databind.ObjectMapper
import org.apache.log4j.LogManager
import org.apache.spark.{SparkConf, SparkContext}
import org.apache.spark.broadcast.Broadcast
import org.apache.spark.sql.SparkSession
import org.apache.spark.streaming.{Duration, Seconds, StreamingContext}
object StreamingApp extends scala.Serializable {
#transient private final val mapper = new ObjectMapper
#transient private final val LOG = LogManager.getLogger(getClass)
#transient private final val executor = Executors.newFixedThreadPool(Runtime.getRuntime.availableProcessors)
var s3 = "s3"
private var shutdownMarker: String = _
private var stopFlag: Boolean = false
def main(args: Array[String]): Unit = {
val queueName = args(0)
val region = args(1)
val fetchMaxMessage = args(2).toInt
val visibilityTimeOutSeconds = args(3).toInt
val waitTimeoutInMillis = args(4).toLong
val isLocal = args(5).toBoolean
val bucket = args(6)
if (args.length >= 10)
shutdownMarker = args(9)
val sparkConf = initialiseSparkConf(isLocal)
sparkConf.set(Constants.QUEUE_NAME, queueName)
sparkConf.set(Constants.REGION, region)
sparkConf.set(Constants.FETCH_MAX_MESSAGE, fetchMaxMessage.toString)
sparkConf.set(Constants.VISIBILITY_TIMEOUT_SECONDS, visibilityTimeOutSeconds.toString)
sparkConf.set(Constants.WAIT_TIMEOUT_IN_MILLIS, waitTimeoutInMillis.toString)
shutdownMarker = s"$s3://$bucket/streaming/shutdownmarker"
val checkpointDirectory = s"$s3://$bucket/streaming/checkpoint/"
var context: StreamingContext = null
try {
context = StreamingContext.getOrCreate(checkpointDirectory, () => createContext(sparkConf, waitTimeoutInMillis, checkpointDirectory, args))
context.start
val checkIntervalMillis = 10000
var isStopped = false
while (!isStopped) {
println("calling awaitTerminationOrTimeout")
isStopped = context.awaitTerminationOrTimeout(checkIntervalMillis)
if (isStopped)
println("confirmed! The streaming context is stopped. Exiting application...")
checkShutdownMarker(context.sparkContext)
if (!isStopped && stopFlag) {
println("stopping ssc right now")
context.stop(stopSparkContext = true, stopGracefully = true)
println("ssc is stopped!!!!!!!")
}
}
}
finally {
LOG.info("Exiting the Application")
if (context != null && org.apache.spark.streaming.StreamingContextState.STOPPED != context.getState) {
context.stop(stopSparkContext = true, stopGracefully = true)
}
if (!executor.isShutdown)
executor.shutdown()
}
}
def checkShutdownMarker(sparkContext: SparkContext): Unit = {
if (!stopFlag) {
stopFlag = isFileExists(shutdownMarker, sparkContext)
}
println(s"Stop marker $shutdownMarker file found: $stopFlag at time ${System.currentTimeMillis()}")
}
def isFileExists(path: String, sparkContext: SparkContext): Boolean = {
isValidPath(isDir = false, path, getFileSystem(path,sparkContext))
}
def getFileSystem(path: String, sparkContext: SparkContext): FileSystem = {
FileSystem.get(URI.create(path), sparkContext.hadoopConfiguration)
}
def isValidPath(isDir: Boolean, path: String, fileSystem: FileSystem): Boolean = {
LOG.info("Validating path {}", path)
if (path.startsWith(Constants.S3) || path.startsWith(Constants.HDFS) || path.startsWith(Constants.FILE)) {
val fsPath = new Path(path)
if (isDir) {
fileSystem isDirectory fsPath
} else {
fileSystem isFile fsPath
}
} else {
Files.exists(Paths.get(path))
}
}
def createContext(sparkConf: SparkConf, waitTimeoutInMillis: Long, checkpointDirectory: String, args: Array[String]): StreamingContext = {
val context = new StreamingContext(sparkConf, Duration(waitTimeoutInMillis + 1000))
processMessage(context, args)
context.checkpoint(checkpointDirectory) // set checkpoint directory
context
}
def processMessage(context: StreamingContext, args: Array[String]): Unit = {
val bucket = args(6)
val wgPath = args(7)
var stagingPath = args(8)
val waitTimeoutInMillis = args(4).toLong
if (context != null) {
if (!stagingPath.endsWith("/")) {
stagingPath = s"$stagingPath/"
}
val outputPrefix = s"$s3://$bucket/$stagingPath"
LOG.info(s"Number of cores for driver: ${Runtime.getRuntime.availableProcessors}")
val sparkContext: SparkContext = context.sparkContext
val broadcasts = BroadCaster.getInstance(sparkContext, s"$s3://$bucket/$wgPath")
val input = context.receiverStream(broadcasts(Constants.SQS_RECEIVER).value.asInstanceOf[SQSReceiver])
//input.checkpoint(interval = Seconds(60))
LOG.info(s"Scheduling mode ${sparkContext.getSchedulingMode.toString}")
input.foreachRDD(r => {
val sparkSession = SparkSession.builder.config(r.sparkContext.getConf).getOrCreate()
val messages = r.collect().map(message => mapper.readValue(message, classOf[Message]))
val broadcasts = BroadCaster.getInstance(r.sparkContext, s"$s3://$bucket/$wgPath")
//Application logic
})
}
}
def initialiseSparkConf(local: Boolean): SparkConf = {
val sparkConf = new SparkConf()
.setAppName("Spark Streaming")
.set("spark.scheduler.mode", "FAIR")
.set("spark.sql.parquet.filterpushdown", "true")
.set("spark.executor.hearbeatInterval", "20")
.set("spark.streaming.driver.writeAheadLog.closeFileAfterWrite", "true")
.set("spark.streaming.receiver.writeAheadLog.closeFileAfterWrite", "true")
.set("spark.streaming.receiver.writeAheadLog.enable", "true")
.set("spark.streaming.stopGracefullyOnShutdown", "true")
.set("spark.streaming.backpressure.enabled","true")
.set("spark.streaming.backpressure.pid.minRate","10") //SQS support batch of 10
if (local) {
s3 = "s3a"
sparkConf.setMaster("local[*]")
} else {
sparkConf.set("hive.metastore.client.factory.class",
"com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory")
}
}
}
object BroadCaster {
#volatile private var instance: Map[String, Broadcast[Any]] = _
def getInstance(sparkContext: SparkContext, wgPath: String): Map[String, Broadcast[Any]] = {
if (instance == null) {
synchronized {
if (instance == null) {
instance = Utils.createBroadcastObjects(wgPath, sparkContext)
instance += (Constants.SQS_RECEIVER -> sparkContext.broadcast(getSQSReceiver(sparkContext.getConf)))
}
}
}
instance
}
private def getSQSReceiver(conf: SparkConf): SQSReceiver = {
val javaSQSReceiver = new SQSReceiver(conf.get(Constants.QUEUE_NAME)).withRegion(Regions.fromName(conf.get(Constants.REGION))).withCredential(new DefaultAWSCredentialsProviderChain())
.withFetchMaxMessage(conf.getInt(Constants.FETCH_MAX_MESSAGE, 10)).withVisibilityTimeOutSeconds(conf.getInt(Constants.VISIBILITY_TIMEOUT_SECONDS, 1800)).withWaitTimeoutinMillis(conf.getLong(Constants.WAIT_TIMEOUT_IN_MILLIS, 1000))
javaSQSReceiver
}
}
import java.util.List;
import org.apache.log4j.Logger;
import org.apache.spark.storage.StorageLevel;
import org.apache.spark.streaming.receiver.Receiver;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.sqs.AmazonSQS;
import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
import com.amazonaws.services.sqs.model.DeleteMessageBatchRequest;
import com.amazonaws.services.sqs.model.DeleteMessageRequest;
import com.amazonaws.services.sqs.model.Message;
import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class SQSReceiver extends Receiver<String> {
private String queueName;
private transient AWSCredentialsProvider credential;
private Regions region = Regions.US_EAST_1;
private Long waitTimeoutinMillis = 0L;
private ObjectMapper mapper= new ObjectMapper();
private transient Logger logger = Logger.getLogger(SQSReceiver.class);
private boolean deleteOnReceipt = false;
private int fetchMaxMessage = 100;
private int visibilityTimeOutSeconds = 60;
private String sqsQueueUrl;
private transient AmazonSQS amazonSQS;
public SQSReceiver(String queueName) {
this(queueName, false);
}
public SQSReceiver(String queueName, boolean deleteOnReceipt) {
super(StorageLevel.MEMORY_AND_DISK_SER());
this.queueName = queueName;
this.deleteOnReceipt = deleteOnReceipt;
setupSQS(queueName);
}
private void setupSQS(String queueName) {
AmazonSQSClientBuilder amazonSQSClientBuilder = AmazonSQSClientBuilder.standard();
if (credential != null) {
amazonSQSClientBuilder.withCredentials(credential);
}
amazonSQSClientBuilder.withRegion(region);
amazonSQS = amazonSQSClientBuilder.build();
sqsQueueUrl = amazonSQS.getQueueUrl(queueName).getQueueUrl();
}
public void onStart() {
new Thread(this::receive).start();
}
public void onStop() {
// There is nothing much to do as the thread calling receive()
// is designed to stop by itself if isStopped() returns false
}
private void receive() {
try {
setupSQS(queueName);
ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest(sqsQueueUrl).withMaxNumberOfMessages(fetchMaxMessage).withVisibilityTimeout(visibilityTimeOutSeconds)
.withWaitTimeSeconds(20); //https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-sqs-long-polling.html
receiveMessagesFromSQS(amazonSQS, sqsQueueUrl, receiveMessageRequest);
} catch (Throwable e) {
stop("Error encountered while initializing", e);
}
}
private void receiveMessagesFromSQS(final AmazonSQS amazonSQS, final String sqsQueueUrl,
ReceiveMessageRequest receiveMessageRequest) {
try {
while (!isStopped()) {
List<Message> messages = amazonSQS.receiveMessage(receiveMessageRequest).getMessages();
if (deleteOnReceipt) {
String receiptHandle = messages.get(0).getReceiptHandle();
messages.forEach(m -> store(m.getBody()));
amazonSQS.deleteMessage(new DeleteMessageRequest(sqsQueueUrl, receiptHandle));
} else {
messages.forEach(this::storeMessage);
}
if (waitTimeoutinMillis > 0L)
Thread.sleep(waitTimeoutinMillis);
}
restart("Trying to connect again");
} catch (IllegalArgumentException | InterruptedException e) {
restart("Could not connect", e);
} catch (Throwable e) {
restart("Error Receiving Data", e);
}
}
private void storeMessage(Message m) {
try {
if (m != null)
store(mapper.writeValueAsString(m));
} catch (JsonProcessingException e) {
logger.error("Unable to write message to streaming context");
}
}
public SQSReceiver withVisibilityTimeOutSeconds(int visibilityTimeOutSeconds) {
this.visibilityTimeOutSeconds = visibilityTimeOutSeconds;
return this;
}
public SQSReceiver withFetchMaxMessage(int fetchMaxMessage) {
if (fetchMaxMessage > 10) {
throw new IllegalArgumentException("FetchMaxMessage can't be greater than 10");
}
this.fetchMaxMessage = fetchMaxMessage;
return this;
}
public SQSReceiver withWaitTimeoutinMillis(long waitTimeoutinMillis) {
this.waitTimeoutinMillis = waitTimeoutinMillis;
return this;
}
public SQSReceiver withRegion(Regions region) {
this.region = region;
return this;
}
public SQSReceiver withCredential(AWSCredentialsProvider credential) {
this.credential = credential;
return this;
}
public void deleteMessages(DeleteMessageBatchRequest request) {
request.withQueueUrl(sqsQueueUrl);
amazonSQS.deleteMessageBatch(request);
}
}

how to fix conversion error when converting from java to kotlin

I am a student, new to kotlin, so I am converting java codes to kotlin to learn and see how it works, but I didnt understand what the error says.
private val _songs = ArrayList<SongInfo>()
internal lateinit var recyclerView: RecyclerView
internal lateinit var seekBar: SeekBar
internal lateinit var songAdapter: SongAdapter
internal var mediaPlayer: MediaPlayer? = null
private val myHandler = Handler()
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
recyclerView = findViewById(R.id.recyclerView) as RecyclerView
seekBar = findViewById(R.id.seekBar) as SeekBar
songAdapter = SongAdapter(this, _songs)
recyclerView.adapter = songAdapter
val linearLayoutManager = LinearLayoutManager(this)
val dividerItemDecoration = DividerItemDecoration(recyclerView.context,
linearLayoutManager.orientation)
recyclerView.layoutManager = linearLayoutManager
recyclerView.addItemDecoration(dividerItemDecoration)
songAdapter.setOnItemClickListener { b, view, obj, position ->
if (b.text == "Stop") {
mediaPlayer!!.stop()
mediaPlayer!!.reset()
mediaPlayer!!.release()
mediaPlayer = null
b.text = "Play"
} else {
val runnable = Runnable {
try {
mediaPlayer = MediaPlayer()
mediaPlayer!!.setDataSource(obj.songUrl)
mediaPlayer!!.prepareAsync()
mediaPlayer!!.setOnPreparedListener { mp ->
mp.start()
seekBar.progress = 0
seekBar.max = mediaPlayer!!.duration
Log.d("Prog", "run: " + mediaPlayer!!.duration)
}
b.text = "Stop"
} catch (e: Exception) {
}
}
myHandler.postDelayed(runnable, 100)
}
}
checkUserPermission()
val t = runThread()
t.start()
}
inner class runThread : Thread() {
override fun run() {
while (true) {
try {
Thread.sleep(1000)
} catch (e: InterruptedException) {
e.printStackTrace()
}
Log.d("Runwa", "run: " + 1)
if (mediaPlayer != null) {
seekBar.post { seekBar.progress = mediaPlayer!!.currentPosition }
Log.d("Runwa", "run: " + mediaPlayer!!.currentPosition)
}
}
}
}
private fun checkUserPermission() {
if (Build.VERSION.SDK_INT >= 23) {
if (ActivityCompat.checkSelfPermission(this, Manifest.permission.READ_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) {
requestPermissions(arrayOf(Manifest.permission.READ_EXTERNAL_STORAGE), 123)
return
}
}
loadSongs()
}
override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<String>, grantResults: IntArray) {
when (requestCode) {
123 -> if (grantResults[0] == PackageManager.PERMISSION_GRANTED) {
loadSongs()
} else {
Toast.makeText(this, "Permission Denied", Toast.LENGTH_SHORT).show()
checkUserPermission()
}
else -> super.onRequestPermissionsResult(requestCode, permissions, grantResults)
}
}
private fun loadSongs() {
val uri = MediaStore.Audio.Media.EXTERNAL_CONTENT_URI
val selection = MediaStore.Audio.Media.IS_MUSIC + "!=0"
val cursor = contentResolver.query(uri, null, selection, null, null)
if (cursor != null) {
if (cursor.moveToFirst()) {
do {
val name = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.DISPLAY_NAME))
val artist = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.ARTIST))
val url = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.DATA))
val s = SongInfo(name, artist, url)
_songs.add(s)
} while (cursor.moveToNext())
}
cursor.close()
songAdapter = SongAdapter(this#MainActivity, _songs)
}
}
}
This is the error:
"Error:(46, 44) Type mismatch: inferred type is (???, ???, ???, ???)
-> Any but SongAdapter.OnItemClickListener was expected Error:(46, 46) Cannot infer a type for this parameter. Please specify it explicitly."
Batch conversion to Kotlin is not the best way to learn the language. I suggest you to re-implement your Android component in Kotlin manually, to get the feeling of language.
The error you see says: "I can not understand how this lambda with 4 parameters can be an instance of SongAdapter.OnItemClickListener, please help". You can try anonymous class in this place.

Could not find matching constructor even when it does

import groovy.util.slurpersupport.Node
import groovy.util.slurpersupport.NodeChild
/**
* Created by on 2017-03-25.
*/
File file = new File("/Users/user1/output/1")
XMLLoader xmlLoader = new XMLLoader(file);
HashMap<String, XmlSlurper> temp = xmlLoader.sendNodes()
JunitDataParser junitParser = new JunitDataParser(temp)
junitParser.populateStats()
public class XMLLoader {
HashMap<String, XmlSlurper> rootNodes; // Collection of root nodes for Junit xml files
public XMLLoader(File junitFile) {
rootNodes = new ArrayList<>()
// If file is directory then closure will search directory to find valid xml files
if (junitFile.isDirectory()) {
junitFile.eachFile {
if (validateFile(it)) {
rootNodes.put(it.name, new XmlSlurper().parse(it)) // Load each fine into arraylist of XMLSlurpers
}
}
} else {
if (validateFile(junitFile)) {
rootNodes.put(junitFile.name, new XmlSlurper().parse(it))
// Load single fine into arraylist of XMLSlurpers
}
}
}
boolean validateFile(File checkFile) {
if (checkFile.name.endsWith('.xml')) {
assert checkFile.exists(): "The file entered could not be be read"
assert checkFile.canRead(): "The file entered could not be be read"
return true;
}
return false
}
HashMap<String, XmlSlurper> sendNodes() {
return rootNodes
}
}
public class JunitDataParser extends GeneralDataParser {
def JunitDataParser(HashMap<String, XmlSlurper> rootNodes) {
super(rootNodes)
}
HashMap<String, ArrayList<TestSuites>> getPopulatedData() {
return super.getPopulatedData()
}
void populateStats() {
ArrayList<TestSuite> stats = getTestSuites();
}
ArrayList<TestSuite> getTestSuites() {
ArrayList<TestSuite> testSuites = new ArrayList<>()
rootNodes.get("first.xml").'**'.findAll {
(it.name().equals("testsuite")) && (it.children().getAt(0).name().equals("testcase"))
}.each {
TestSuite temp = new TestSuite(it.attributes().get("name"), it.attributes().get("tests").toInteger(), it.attributes().get("errors").toInteger(), it.attributes().get("time"), it.attributes().get("timestamp"), getTestCases(it))
testSuites.add(temp)
}
return testSuites
}
ArrayList<TestCase> getTestCases(NodeChild testSuiteNode) {
ArrayList<TestCase> testCases = new ArrayList<>()
testSuiteNode.childNodes().each {
String time;
if (time!=null && time.length()>0) {
time = it.attributes().get("time")
} else {
time = "Time Not Found"
}
testCases.add(new TestCase(it.attributes().get("name"), getOutCome(it), time))
}
return testCases
}
TestResult getOutCome(Node testCaseNode) {
String testMessage = "hi"
int testOutcome = 0;
if (testCaseNode.childNodes().size() != 0) {
testCaseNode.childNodes().each {
it.attributes().each {
testMessage = testMessage + "\n $it.key : $it.value"
}
}
testOutcome =1
}
TestResult temp = new TestResult(0,testMessage)
return temp
}
}
// The Base abstract class that all additional parsers should extend and populate
public abstract class GeneralDataParser {
HashMap<String, XmlSlurper> rootNodes
HashMap<String, ArrayList<TestSuites>> populatedData
public GeneralDataParser(HashMap<String, XmlSlurper> rootNodes) {
this.rootNodes = rootNodes
}
abstract void populateStats()
abstract ArrayList<TestSuite> getTestSuites()
abstract ArrayList<TestCase> getTestCases(NodeChild testSuiteNodes)
abstract TestResult getOutCome(Node testCaseNode)
}
public class TestSuites {
ArrayList<TestSuite> testSuites
TestSuites(ArrayList<TestSuite> testSuites) {
this.testSuites = testSuites
}
}
public class TestSuite {
String testSuiteName
int numOfTests
int numOfErrors
int numOfFailures
String timeTaken;
String timeStamp;
ArrayList<TestCase> testCases
TestSuite(String testSuiteName, int numOfTests, int numOfErrors, int numOfFailures, String timeTaken, String timeStamp, ArrayList<TestCase> testCases) {
this.testSuiteName = testSuiteName
this.numOfTests = numOfTests
this.numOfErrors = numOfErrors
this.numOfFailures = numOfFailures
this.timeTaken = timeTaken
this.timeStamp = timeStamp
this.testCases = testCases
}
}
public class TestCase {
String testCasename
TestResult testOutcome
String testCaseTime = ""
TestCase(String testCasename, TestResult testOutcome, String testCaseTime) {
this.testCasename = testCasename
this.testOutcome = testOutcome
this.testCaseTime = testCaseTime
}
}
public class TestResult {
int valOfResult
String textOfResult
TestResult(int valOfResult, String textOfResult) {
this.valOfResult = valOfResult
this.textOfResult = textOfResult
}
}
I keep getting a matching constructor could not be found can anyone see any issues?
Caught: groovy.lang.GroovyRuntimeException: Could not find matching constructor for: TestSuite(java.lang.String, java.lang.Integer, java.lang.Integer, java.lang.String, java.lang.String, java.util.ArrayList)
groovy.lang.GroovyRuntimeException: Could not find matching constructor for: TestSuite(java.lang.String, java.lang.Integer, java.lang.Integer, java.lang.String, java.lang.String, java.util.ArrayList)
at JunitDataParser$_getTestSuites_closure2.doCall(JunitXMLToBeautifulHTML.groovy:70)
at JunitDataParser.getTestSuites(JunitXMLToBeautifulHTML.groovy:67)
at JunitDataParser.populateStats(JunitXMLToBeautifulHTML.groovy:62)
at JunitDataParser$populateStats.call(Unknown Source)
at JunitXMLToBeautifulHTML.run(JunitXMLToBeautifulHTML.groovy:13)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
The error message is correct.
The constructor's signature is:
TestSuite(String testSuiteName, int numOfTests, int numOfErrors, int numOfFailures, String timeTaken, String timeStamp, ArrayList<TestCase> testCases)
And you're trying to use:
new TestSuite(it.attributes().get("name"), it.attributes().get("tests").toInteger(), it.attributes().get("errors").toInteger(), it.attributes().get("time"), it.attributes().get("timestamp"), getTestCases(it))
So you're not passing a numOfFailures.

Resources