I was going through the Chisel 2.2 Tutorial manual (I am aware that Chisel3 is out in BETA version, but I am required to use Chisel2.2 for some extension of previously implemented modules).
I have been looking for examples of using DecoupledIO interface in Chisel and found a few in the tutorial mentioned above and also on StackOverflow.
One such example is Parameterized FIFO example in Chisel Tutorial manual whose implementation is:
class Fifo[T <: Data] (type: T, n: Int)extends Module {
val io = new Bundle {
val enq_val = Bool(INPUT)
val enq_rdy = Bool(OUTPUT)
val deq_val = Bool(OUTPUT)
val deq_rdy = Bool(INPUT)
val enq_dat = type.asInput
val deq_dat = type.asOutput
}
val enq_ptr= Reg(init = UInt(0, sizeof(n)))
val deq_ptr= Reg(init = UInt(0, sizeof(n)))
val is_full= Reg(init = Bool(false))
val do_enq= io.enq_rdy && io.enq_val
val do_deq= io.deq_rdy && io.deq_val
val is_empty= !is_full && (enq_ptr === deq_ptr)
val deq_ptr_inc = deq_ptr + UInt(1)
val enq_ptr_inc = enq_ptr + UInt(1)
val is_full_next = Mux(do_enq && ~do_deq &&(enq_ptr_inc===deq_ptr),Bool(true),Mux(do_deq && is_full, Bool(false), is_full))
enq_ptr := Mux(do_enq, enq_ptr_inc, enq_ptr)
deq_ptr := Mux(do_deq, deq_ptr_inc, deq_ptr)
is_full := is_full_next
val ram = Mem(n)
when (do_enq) {
ram(enq_ptr) := io.enq_dat
}
io.enq_rdy := !is_full
io.deq_val := !is_empty
ram(deq_ptr) <> io.deq_dat
}
I understand most of the implementation but the code snippet:
val is_full_next = Mux(do_enq && ~do_deq &&(enq_ptr_inc===deq_ptr),Bool(true),Mux(do_deq && is_full, Bool(false), is_full))
If I am interpreting it correctly we are trying to check if the next operation will lead to the condition of isFull or not. If it is so, why are we looking to check the value of enq_ptr_inc === deq_ptr.
If someone can share their view on how this might be working, I would like to hear from you.
Also, I am not sure if this is the simplest way to implement a parameterized FIFO. I am working on my own implementation, but if there is an easier way to implement a FIFO (not even a parameterized one), it would help me to clear my doubts.
Related
I have a string that is a version number but I only need the last two digits of the string. I.e, 15.0.4571.1502 -> 4571.1502. I can't seem to figure out an efficient way to do this in Kotlin.
Code:
version = "15.0.4571.1502"
var buildOne = version.split(".").toTypedArray()[2]
var buildTwo = version.split(".").toTypedArray()[3]
var new = "$BuildOne"."$BuildTwo"
Error:
The expression cannot be a selector (occur after a dot)
You've written the dot outside of the string template, the following should work:
val new = "$BuildOne.$BuildTwo"
You can further simplify your solution, by making use of functions provided in the Kotlin standard library.
val version = "15.0.4571.1502"
val new = version
.split(".")
.takeLast(2)
.joinToString(".")
val version = "15.0.4571.1502"
val new = version.split('.').drop(2).joinToString(".")
// also possible:
// val new = version.split('.').takeLast(2).joinToString(".")
println(new)
Another possible solution using List destructuring:
val version = "15.0.4571.1502"
val (_, _, buildOne, buildTwo) = version.split(".")
val new = "$buildOne.$buildTwo"
I am testing this function. The main bit for me is the call to add method of a respository (partitionsOfATagTransactionRepository.add(transaction, infoToAdd,mutationCondition))
def updateOrCreateTagPartitionInfo(transaction:DistributedTransaction,currentTagPartition: Option[TagPartitions], tag: String) = {
val currentCalendar = Calendar.getInstance() //TODOM - should I use a standard Locale/Timezone (eg GMT) to keep time consistent across all instances of the server application
val currentYear = currentCalendar.get(Calendar.YEAR).toLong
val currentMonth = currentCalendar.get(Calendar.MONTH).toLong
val newTagParitionInfo = TagPartitionsInfo(currentYear.toLong, currentMonth.toLong)
val (infoToAdd,mutationCondition) = currentTagPartition match {
case Some(tagPartitionInfo) => {
//checktest-should add new tag partition info to existing partition info
(TagPartitions(tagPartitionInfo.tag, tagPartitionInfo.partitionInfo + (newTagParitionInfo)),new PutIfExists)
}
case None => {
//checktest-should add new tag partition info if existing partition doesn't exist
(TagPartitions(tag, Set(newTagParitionInfo)),new PutIfNotExists)
}
}
partitionsOfATagTransactionRepository.add(transaction, infoToAdd,mutationCondition) //calling a repositoru method which I suppose needs mocking
infoToAdd
}
I wrote this test case to test the method
"should add new tag partition info if existing partition doesn't exist" in {
val servicesTestEnv = new ServicesTestEnv(components = components)
val questionTransactionDBService = new QuestionsTransactionDatabaseService(
servicesTestEnv.mockAnswersTransactionRepository,
servicesTestEnv.mockPartitionsOfATagTransactionRepository,
servicesTestEnv.mockPracticeQuestionsTagsTransactionRepository,
servicesTestEnv.mockPracticeQuestionsTransactionRepository,
servicesTestEnv.mockSupportedTagsTransactionRepository,
servicesTestEnv.mockUserProfileAndPortfolioTransactionRepository,
servicesTestEnv.mockQuestionsCreatedByUserRepo,
servicesTestEnv.mockTransactionService,
servicesTestEnv.mockPartitionsOfATagRepository,
servicesTestEnv.mockHelperMethods
)
val currentCalendar = Calendar.getInstance() //TODOM - should I use a standard Locale/Timezone (eg GMT) to keep time consistent across all instances of the server application
val currentYear = currentCalendar.get(Calendar.YEAR).toLong
val currentMonth = currentCalendar.get(Calendar.MONTH).toLong
val newTagParitionInfo = TagPartitionsInfo(currentYear.toLong, currentMonth.toLong)
val existingTag = "someExistingTag"
val existingTagPartitions = None
val result = questionTransactionDBService.updateOrCreateTagPartitionInfo(servicesTestEnv.mockDistributedTransaction,
existingTagPartitions,existingTag) //calling the funtion under test but have not provided mock for the repository's add method. The test passes! how? Shouldn't the test throw Null Pointer exception?
val expectedResult = TagPartitions(existingTag,Set(newTagParitionInfo))
verify(servicesTestEnv.mockPartitionsOfATagTransactionRepository,times(1))
.add(servicesTestEnv.mockDistributedTransaction,expectedResult,new PutIfNotExists())
result mustBe expectedResult
result mustBe TagPartitions(existingTag,Set(newTagParitionInfo))
}
The various mocks are defined as
val mockCredentialsProvider = mock(classOf[CredentialsProvider])
val mockUserTokenTransactionRepository = mock(classOf[UserTokenTransactionRepository])
val mockUserTransactionRepository = mock(classOf[UserTransactionRepository])
val mockUserProfileAndPortfolioTransactionRepository = mock(classOf[UserProfileAndPortfolioTransactionRepository])
val mockHelperMethods = mock(classOf[HelperMethods])
val mockTransactionService = mock(classOf[TransactionService])
val mockQuestionsCreatedByUserRepo = mock(classOf[QuestionsCreatedByAUserForATagTransactionRepository])
val mockQuestionsAnsweredByUserRepo = mock(classOf[QuestionsAnsweredByAUserForATagTransactionRepository])
val mockDistributedTransaction = mock(classOf[DistributedTransaction])
val mockQuestionTransactionDBService = mock(classOf[QuestionsTransactionDatabaseService])
val mockQuestionNonTransactionDBService = mock(classOf[QuestionsNonTransactionDatabaseService])
val mockAnswersTransactionRepository = mock(classOf[AnswersTransactionRepository])
val mockPartitionsOfATagTransactionRepository = mock(classOf[PartitionsOfATagTransactionRepository])
val mockPracticeQuestionsTagsTransactionRepository = mock(classOf[PracticeQuestionsTagsTransactionRepository])
val mockPracticeQuestionsTransactionRepository = mock(classOf[PracticeQuestionsTransactionRepository])
val mockSupportedTagsTransactionRepository = mock(classOf[SupportedTagsTransactionRepository])
val mockPartitionsOfATagRepository = mock(classOf[PartitionsOfATagRepository])
The test case passes even though I have not provided any mock for partitionsOfATagTransactionRepository.add. Should I get a NullPointer exception when the add method is called.
I was expecting that I would need to write something like doNothing().when(servicesTestEnv.mockPartitionsOfATagTransactionRepository).add(ArgumentMatchers.any[DistributedTransaction],ArgumentMatchers.any[TagPartitions],ArgumentMatchers.any[MutationCondition]) or when(servicesTestEnv.mockPartitionsOfATagTransactionRepository).add(ArgumentMatchers.any[DistributedTransaction],ArgumentMatchers.any[TagPartitions],ArgumentMatchers.any[MutationCondition]).thenReturn(...) for the test case to pass.
Mockito team made a decision to return default value for a method if no stubbing is provided.
See: https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html#stubbing
By default, for all methods that return a value, a mock will return either null, a primitive/primitive wrapper value, or an empty collection, as appropriate. For example 0 for an int/Integer and false for a boolean/Boolean.
This decision was made consciously: if you are focusing on a different aspect of behaviour of method under test, and the default value is good enough, you don't need to specify it.
Note that other mocking frameworks have taken opposite path - they raise an exception when unstubbed call is detected (for example: EasyMock).
See EasyMock vs Mockito: design vs maintainability?
The following code runs into future timeouts (in Scala 2.x and Dotty, -Xcheckinit or -Ycheck-init does not help here) because of cyclic object initialization. In complex projects these cycles usually are hidden very well. Is there any possiblity of getting help from the compiler or at least at runtime? How do you prevent this from happening in a multithreaded environment?
import scala.concurrent.Future
import scala.concurrent._
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
object Base {
val LeftElement = "Left"
val RightElement = "Right"
println("Base before: " + Thread.currentThread())
val all = Set(Left, Right)
println("Base after: " + Thread.currentThread())
}
object Left {
println("Left before: " + Thread.currentThread())
val basePath = Base.LeftElement
}
object Right {
println("Right before: " + Thread.currentThread())
val basePath = Base.RightElement
}
object Main extends App {
val f1 = Future(Left)
val f2 = Future(Right)
println(Await.result(f1, 1 second))
println(Await.result(f2, 1 second))
}
In general, the compiler and JVM will not help you avoid this.
The best you can do to address this is delay evaluation of the cycle by, for instance:
using lazy val
using def
Note that either results in some overhead relative to a simple val. I haven't done experiments, but I'd suspect that lazy val (incurring the expense of some synchronization) is better for a case like
lazy val all = Set(Left, Right)
to limit allocations of redundant objects, and that def is better for a case like
def basePath = Base.LeftElement
since that's pretty likely to be inlined by JIT.
See also: How to diagnose or detect deadlocks in static initializers
I am new to Scala but I know Java. Thus, as far as I understand, the difference is that == in Scala acts as .equals in Java, which means we are looking for the value; and eq in Scala acts as == in Java, which means we are looking for the reference address and not value.
However, after running the code below:
val greet_one_v1 = "Hello"
val greet_two_v1 = "Hello"
println(
(greet_one_v1 == greet_two_v1),
(greet_one_v1 eq greet_two_v1)
)
val greet_one_v2 = new String("Hello")
val greet_two_v2 = new String("Hello")
println(
(greet_one_v2 == greet_two_v2),
(greet_one_v2 eq greet_two_v2)
)
I get the following output:
(true,true)
(true,false)
My theory is that the initialisation of these strings differs. Hence, how is val greet_one_v1 = "Hello" different from val greet_one_v2 = new String("Hello")? Or, if my theory is incorrect, why do I have different outputs?
As correctly answered by Luis Miguel Mejía Suárez, the answer lies in String Interning which is part of what JVM (Java Virtual Machine) does automatically. To initiate a new String it needs to be initiated explicitly like in my example above; otherwise, Java will allocate the same memory for same values for optimisation reasons.
I need to correct some spellings using spark.
Unfortunately a naive approach like
val misspellings3 = misspellings1
.withColumn("A", when('A === "error1", "replacement1").otherwise('A))
.withColumn("A", when('A === "error1", "replacement1").otherwise('A))
.withColumn("B", when(('B === "conditionC") and ('D === condition3), "replacementC").otherwise('B))
does not work with spark How to add new columns based on conditions (without facing JaninoRuntimeException or OutOfMemoryError)?
The simple cases (the first 2 examples) can nicely be handled via
val spellingMistakes = Map(
"error1" -> "fix1"
)
val spellingNameCorrection: (String => String) = (t: String) => {
titles.get(t) match {
case Some(tt) => tt // correct spelling
case None => t // keep original
}
}
val spellingUDF = udf(spellingNameCorrection)
val misspellings1 = hiddenSeasonalities
.withColumn("A", spellingUDF('A))
But I am unsure how to handle the more complex / chained conditional replacements in an UDF in a nice & generalizeable manner.
If it is only a rather small list of spellings < 50 would you suggest to hard code them within a UDF?
You can make the UDF receive more than one column:
val spellingCorrection2= udf((x: String, y: String) => if (x=="conditionC" && y=="conditionD") "replacementC" else x)
val misspellings3 = misspellings1.withColumn("B", spellingCorrection2($"B", $"C")
To make this more generalized you can use a map from a tuple of the two conditions to a string same as you did for the first case.
If you want to generalize it even more then you can use dataset mapping. Basically create a case class with the relevant columns and then use as to convert the dataframe to a dataset of the case class. Then use the dataset map and in it use pattern matching on the input data to generate the relevant corrections and convert back to dataframe.
This should be easier to write but would have a performance cost.
For now I will go with the following which seems to work just fine and is more understandable: https://gist.github.com/rchukh/84ac39310b384abedb89c299b24b9306
If spellingMap is the map containing correct spellings, and df is the dataframe.
val df: DataFrame = _
val spellingMap = Map.empty[String, String] //fill it up yourself
val columnsWithSpellingMistakes = List("abc", "def")
Write a UDF like this
def spellingCorrectionUDF(spellingMap:Map[String, String]) =
udf[(String), Row]((value: Row) =>
{
val cellValue = value.getString(0)
if(spellingMap.contains(cellValue)) spellingMap(cellValue)
else cellValue
})
And finally, you can call them as
val newColumns = df.columns.map{
case columnName =>
if(columnsWithSpellingMistakes.contains(columnName)) spellingCorrectionUDF(spellingMap)(Column(columnName)).as(columnName)
else Column(columnName)
}
df.select(newColumns:_*)