I currently transpile the control flow modeled in an SCXML state-chart onto an ANSI-C algorithm which calls a series of user-supplied callback functions in the correct order, effectively realizing the control flow from the state-chart or ANSI-C. Seeing that more target languages may eventually follow, I was thinking about transpiling onto Haxe as a quasi-canonical form and use their transpilation capabilities to target other languages.
Seeing that Haxe is inherently object-oriented, I guess the best way would be to generate an abstract base class with the transpiled algorithm, which would be extended with implementations of the callbacks.
However, looking at Haxe it seems that this is a rather unorthodox usage and I am at a loss how best to approach it. I cannot find native callbacks in the target language agnostic part of Haxe, so I guess it boils down to target language specific approaches anyway?
Update: I want to invoke user-supplied callbacks in the target language. The state-chart here merely formalizes a certain control flow. There is no XML parsing involved in Haxe at all, I already parse the XML, process it and generate ANSI-C which accepts user-supplied callbacks. Now I want to take a detour via Haxe to generate any target-language, still, the user-supplied callbacks and all the "scaffolding" is in the target-language.
If you only need one listener per callback then just use a function per callback, I prefer to not anticipate what data the listener needs.
Run js example here: https://try.haxe.org/#2a6d2
code below for completeness.
class Test {
static function main() { new Test(); }
var testing: WithCallback;
public function new(){
testing = new WithCallback( output );
testing.start();
}
public function output(){
trace( 'test ' + testing.val );
}
}
class WithCallback{
public var cb: Void->Void;
public var val: String;
public function new( cb_: Void -> Void ){
cb = cb_;
}
public function start(){
for( i in 0...100 ){
val = 'callback counting ' + Std.string( i );
if( cb != null ) cb();
}
}
}
If you want multiple objects to listen then you could look at some Signal type library, I believe Tink ( macro library ) provides one but not tried it.
https://github.com/haxetink/tink_core/blob/master/src/tink/core/Signal.hx
There must be a few signals libraries around.
https://code.google.com/archive/p/hxs/
https://github.com/massiveinteractive/msignal
You may also want to look at how Json can be autoparsed in Haxe with abstracts and typedef using stuff like '#to' and '#from'.
https://haxe.org/manual/std-Json-parsing.html
So for instance a nice way to parse some json with Time field in - is to parse to a typedef with the time field using an abstract around a string and add a method within the abstract so you can get the type from the abstract.
https://haxe.org/manual/types-abstract-implicit-casts.html
I think others have worked on a similar approach for xml parsing but if you look into the internals of haxe.Json.parse I am sure you could create a similar approach for xml or binary feeds ( not sure if franco's streams stuff is relevant ). Also there is an approach to get haxe to generate the typedef code for json parsing based on a sample using really smart macros, but I guess it would be very hard to get haxe macros to construct parser based on data example.
Also you should have a look in the format library it has many example of parsing data.
https://github.com/HaxeFoundation/format
Related
I'm writing a grammar for C++ target, however I'd like to keep it working with Java as well since ANTLR comes with great tools that work for grammars with Java target. The book ("The Definitive ANTLR 4 Reference") says that the way of achieving target independence is to use listeners and/or visitors. There is one problem though. Any predicate, local variable, custom constructor, custom token class etc. that I might need introduces target language dependence that cannot be removed, at least according to the information I took from the book. Since the book might be outdated here are the questions:
Is there a way of declaring primitive variables in language independent way, something like:
item[$bool hasAttr]
:
type ( { $hasAttr }? attr | ) ID
;
where $bool would be translated to bool in C++, but to boolean in Java (workaround would be to use int in that case but most likely not in all potential targets)
Is there a way of declaring certain code fragments to be for specific target only, something like:
parser grammar testParser;
options
{
tokenVocab=testLexer;
}
#header
<lang=Cpp>{
#include "utils/helper.h"
}
<lang=Java>{
import test.utils.THelper;
}
#members
<lang=Cpp>{
public:
testParser(antlr4::TokenStream *input, utils::THelper *helper);
private:
utils::THelper *Helper;
public:
}
<lang=Java>{
public testParser(TokenStream input, THelper helper) {
this(input);
Helper = helper;
}
private THelper Helper;
}
start
:
(
<lang=Cpp>{ Helper->OnUnitStart(this); }
<lang=Java>{ Helper.OnUnitStart(this); }
unit
<lang=Cpp>{ _localctx = Helper->OnUnitEnd(this); }
<lang=Java>{ _localctx = Helper.OnUnitEnd(this); }
)*
EOF
;
...
For the time being I'm keeping two separate grammars changing the Java one and merging the changes to C++ one once I'm happy with the results, but if possible
I'd rather keep it in one file.
This target dependency is a real nuisance and I'm thinking for a while already how to get rid of that in a good way. Haven't still found something fully usable.
What you can do is to stay with syntax that both Java and C++ can understand (e.g. write a predicate like a function call: a: { isValid() }? b c; and implement such functions in a base class from which you derive your parser (ANTLR allows to specify such a base class via the grammar option superClass).
The C++ target also got a number of additional named actions which you can use to specify C++ specific stuff only.
I'm doing this Ensime package for Atom.io https://github.com/ensime/ensime-atom and I've been thinking about the possibility to use scala.js instead of writing Coffeescript.
Atom is a web based editor which is scripted with js and is node.js based. A plugin/package defines it's main entry point by pointing out a javascript object with a few specific.
I figured I should start out simple and try using scala.js replacing the simplest coffeescript file I have:
{View} = require 'atom-space-pen-views'
# View for the little status messages down there where messages from Ensime server can be shown
module.exports =
class StatusbarView extends View
#content: ->
#div class: 'ensime-status inline-block'
initialize: ->
serialize: ->
init: ->
#attach()
attach: =>
statusbar = document.querySelector('status-bar')
statusbar?.addLeftTile {item: this}
setText: (text) =>
#text("Ensime: #{text}").show()
destroy: ->
#detach()
As you can see this exports a require.js module and is a class extending a class fetched with require as well.
Sooo.
I'm thinking I'd just use Dynamic for the require dep as I've seen on SO How to invoke nodejs modules from scala.js?:
import js.Dynamic.{global => g}
import js.DynamicImplicits._
private[views] object SpacePen {
private val spacePenViews = require("atom-space-pen-views")
val view = spacePenViews.view
}
But if I wanted to type the super-class, could I just make a facade-trait and do asInstanceOf?
Secondly, I wonder how I can export my class as a node module. I found this:
https://github.com/rockymadden/scala-node/blob/master/main/src/main/coffeescript/example.coffee
Is this the right way? Do I need to do the sandboxing? Couldn't I just get moduleimported from global and write module.exports = _some_scala_object_?
I'm also wondering how I could extend existing js classes. The same problem as asked here, but I don't really understand the answer:
https://groups.google.com/forum/#!topic/scala-js/l0gSOSiqubs
My code so far:
private[views] object SpacePen {
private val spacePenViews = js.Dynamic.global.require("atom-space-pen-views")
type View = spacePenViews.view
}
class StatusBarView extends SpacePen.View {
override def content =
super.div()
}
gives me compile errors that I can't extend sealed trait Dynamic. Of course.
Any pointers highly appreciated!
I'm not particularly expert in Node per se, but to answer your first question, yes -- if you have a pointer to a JS object, and you know the details of its type, you can pretty much always define a facade trait and asInstanceOf to use it. That ought to work.
As for the last bit, you basically can't extend JS classes in Scala.js -- it just doesn't work. The way most of us get around that is by defining implicit classes, or using implicit def's, to get the appearance of extending without actually doing so.
For example, given JS class Foo, I can write
implicit class RichFoo(foo:Foo) {
def method1() = { ... }
}
This is actually a wrapper around Foo, but calling code can simply call foo.method1() without worrying about that detail.
You can see this approach in action very heavily in jquery-facade, particularly in the relationship between JQuery (the pure facade), JQueryTyped (some tweaked methods over JQuery to make them work better in Scala), and JQueryExtensions (some higher-level functions built around JQuery). These are held together using implicit def's in package.scala. As far as calling code is concerned, all of these simply look like methods on JQuery.
Thanks in advance for the help -
I am new to mockito but have spent the last day looking at examples and the documentation but haven't been able to find a solution to my problem, so hopefully this is not too dumb of a question.
I want to verify that deleteLogs() calls deleteLog(Path) NUM_LOGS_TO_DELETE number of times, per path marked for delete. I don't care what the path is in the mock (since I don't want to go to the file system, cluster, etc. for the test) so I verify that deleteLog was called NUM_LOGS_TO_DELETE times with any non-null Path as a parameter. When I step through the execution however, deleteLog gets passed a null argument - this results in a NullPointerException (based on the behavior of the code I inherited).
Maybe I am doing something wrong, but verify and the use of isNotNull seems pretty straight forward...here is my code:
MonitoringController mockController = mock(MonitoringController.class);
// Call the function whose behavior I want to verify
mockController.deleteLogs();
// Verify that mockController called deleteLog the appropriate number of times
verify(mockController, Mockito.times(NUM_LOGS_TO_DELETE)).deleteLog(isNotNull(Path.class));
Thanks again
I've never used isNotNull for arguments so I can't really say what's going wrong with you code - I always use an ArgumentCaptor. Basically you tell it what type of arguments to look for, it captures them, and then after the call you can assert the values you were looking for. Give the below code a try:
ArgumentCaptor<Path> pathCaptor = ArgumentCaptor.forClass(Path.class);
verify(mockController, Mockito.times(NUM_LOGS_TO_DELETE)).deleteLog(pathCaptor.capture());
for (Path path : pathCaptor.getAllValues()) {
assertNotNull(path);
}
As it turns out, isNotNull is a method that returns null, and that's deliberate. Mockito matchers work via side effects, so it's more-or-less expected for all matchers to return dummy values like null or 0 and instead record their expectations on a stack within the Mockito framework.
The unexpected part of this is that your MonitoringController.deleteLog is actually calling your code, rather than calling Mockito's verification code. Typically this happens because deleteLog is final: Mockito works through subclasses (actually dynamic proxies), and because final prohibits subclassing, the compiler basically skips the virtual method lookup and inlines a call directly to the implementation instead of Mockito's mock. Double-check that methods you're trying to stub or verify are not final, because you're counting on them not behaving as final in your test.
It's almost never correct to call a method on a mock directly in your test; if this is a MonitoringControllerTest, you should be using a real MonitoringController and mocking its dependencies. I hope your mockController.deleteLogs() is just meant to stand in for your actual test code, where you exercise some other component that depends on and interacts with MonitoringController.
Most tests don't need mocking at all. Let's say you have this class:
class MonitoringController {
private List<Log> logs = new ArrayList<>();
public void deleteLogs() {
logs.clear();
}
public int getLogCount() {
return logs.size();
}
}
Then this would be a valid test that doesn't use Mockito:
#Test public void deleteLogsShouldReturnZeroLogCount() {
MonitoringController controllerUnderTest = new MonitoringController();
controllerUnderTest.logSomeStuff(); // presumably you've tested elsewhere
// that this works
controllerUnderTest.deleteLogs();
assertEquals(0, controllerUnderTest.getLogCount());
}
But your monitoring controller could also look like this:
class MonitoringController {
private final LogRepository logRepository;
public MonitoringController(LogRepository logRepository) {
// By passing in your dependency, you have made the creator of your class
// responsible. This is called "Inversion-of-Control" (IoC), and is a key
// tenet of dependency injection.
this.logRepository = logRepository;
}
public void deleteLogs() {
logRepository.delete(RecordMatcher.ALL);
}
public int getLogCount() {
return logRepository.count(RecordMatcher.ALL);
}
}
Suddenly it may not be so easy to test your code, because it doesn't keep state of its own. To use the same test as the above one, you would need a working LogRepository. You could write a FakeLogRepository that keeps things in memory, which is a great strategy, or you could use Mockito to make a mock for you:
#Test public void deleteLogsShouldCallRepositoryDelete() {
LogRepository mockLogRepository = Mockito.mock(LogRepository.class);
MonitoringController controllerUnderTest =
new MonitoringController(mockLogRepository);
controllerUnderTest.deleteLogs();
// Now you can check that your REAL MonitoringController calls
// the right method on your MOCK dependency.
Mockito.verify(mockLogRepository).delete(Mockito.eq(RecordMatcher.ALL));
}
This shows some of the benefits and limitations of Mockito:
You don't need the implementation to keep state any more. You don't even need getLogCount to exist.
You can also skip creating the logs, because you're testing the interaction, not the state.
You're more tightly-bound to the implementation of MonitoringController: You can't simply test that it's holding to its general contract.
Mockito can stub individual interactions, but getting them consistent is hard. If you want your LogRepository.count to return 2 until you call delete, then return 0, that would be difficult to express in Mockito. This is why it may make sense to write fake implementations to represent stateful objects and leave Mockito mocks for stateless service interfaces.
(Pseudo-)Code
Here is a non-compilable code-sketch of the concepts I am having trouble with:
struct Data {};
struct A {};
struct B {};
struct C {};
/* and many many more...*/
template<typename T>
class Listener {
public:
Listener(MyObject* worker):worker(worker)
{ /* do some magic to register with RTI DDS */ };
public:
// This function is used ass a callback from RTI DDS, i.e. it will be
// called from other threads when new Data is available
void callBackFunction(Data d)
{
T t = extractFromData(d);
// Option 1: direct function call
// works somewhat, but shows "QObject::startTimer: timers cannot be started
// from another thread" at the console...
worker->doSomeWorkWithData(t); //
// Option 2: Use invokeMethod:
// seems to fail, as the macro expands including '"T"' and that type isn't
// registered with the QMetaType system...
// QMetaObject::invokeMethod(worker,"doSomeGraphicsWork",Qt::AutoConnection,
// Q_ARG(T, t)
// );
// Option 3: use signals slots
// fails as I can't make Listener, a template class, a QObject...
// emit workNeedsToBeDone(t);
}
private:
MyObject* worker;
T extractFromData(Data d){ return T(d);};
};
class MyObject : public QObject {
Q_OBJECT
public Q_SLOTS:
void doSomeWorkWithData(A a); // This one affects some QGraphicsItems.
void doSomeWorkWithData(B b){};
void doSomeWorkWithData(C c){};
public:
MyObject():QObject(nullptr){};
void init()
{
// listeners are not created in the constructor, but they should have the
// same thread affinity as the MyObject instance that creates them...
// (which in this example--and in my actual code--would be the main GUI
// thread...)
new Listener<A>(this);
new Listener<B>(this);
new Listener<C>(this);
};
};
main()
{
QApplication app;
/* plenty of stuff to set up RTI DDS and other things... */
auto myObject = new MyObject();
/* stuff resulting in the need to separate "construction" and "initialization" */
myObject.init();
return app.exec();
};
Some more details from the actual code:
The Listener in the example is a RTI DataReaderListener, the callback
function is onDataAvailable()
What I would like to accomplish
I am trying to write a little distributed program that uses RTI's Connext DDS for communication and Qt5 for the GUI stuff--however, I don't believe those details do matter much as the problem, as far as I understood it, boils down to the following:
I have a QObject-derived object myObject whose thread affinity might or might not be with the main GUI thread (but for simplicity, let's assume that is the case.)
I want that object to react to event's which happen in another, non-Qt 3rd-party library (in my example code above represented by the functions doSomeWorkWithData().
What I understand so far as to why this is problematic
Disclaimer: As usual, there is always more than one new thing one learns when starting a new project. For me, the new things here are/were RTI's Connext and (apparently) my first time where I myself have to deal with threads.
From reading about threading in Qt (1,2,3,4, and 5 ) it seems to me that
QObjects in general are not thread safe, i.e. I have to be a little careful about things
Using the right way of "communicating" with QObjects should allow me to avoid having to deal with mutexes etc myself, i.e. somebody else (Qt?) can take care of serializing access for me.
As a result from that, I can't simply have (random) calls to MyClass::doSomeWorkWithData() but I need to serialize that. One, presumably easy, way to do so is to post an event to the event queue myObject lives in which--when time is available--will trigger the execution of the desired method, MyClass::doSomeWorkWithData() in my case.
What I have tried to make things work
I have confirmed that myObject, when instantiated similarly as in the sample code above, is affiliated with the main GUI thread, i.e. myObject.thread() == QApplication::instance()->thread().
With that given, I have tried three options so far:
Option 1: Directly calling the function
This approach is based upon the fact that
- myObject lives in the GUI thread
- All the created listeners are also affiliated with the GUI thread as they are
created by `myObject' and inherit its thread that way
This actually results in the fact that doSomeWorkWithData() is executed. However,
some of those functions manipulate QGraphicsItems and whenever that is the case I get
error messages reading: "QObject::startTimer: timers cannot be started from another
thread".
Option 2: Posting an event via QMetaObject::invokeMethod()
Trying to circumvent this problem by properly posting an event for myObject, I
tried to mark MyObject::doSomeWorkWithData() with Q_INVOKABLE, but I failed at invoking the
method as I need to pass arguments with Q_ARG. I properly registered and declared my custom types
represented by struct A, etc. in the example), but I failed at the fact the
Q_ARG expanded to include a literal of the type of the argument, which in the
templated case didn't work ("T" isn't a registered or declared type).
Trying to use conventional signals and slots
This approach essentially directly failed at the fact that the QMeta system doesn't
work with templates, i.e. it seems to me that there simply can't be any templated QObjects.
What I would like help with
After spending about a week on attempting to fix this, reading up on threads (and uncovering some other issues in my code), I would really like to get this done right.
As such, I would really appreciate if :
somebody could show me a generic way of how a QObject's member function can be called via a callback function from another 3rd-party library (or anything else for that matter) from a different, non QThread-controlled, thread.
somebody could explain to me why Option 1 works if I simply don't create a GUI, i.e. do all the same work, just without a QGraphcisScene visualizing it (and the project's app being a QCoreApplication instead of a QApplication and all the graphics related work #defineed out).
Any, and I mean absolutely any, straw I could grasp on is truly appreciated.
Update
Based on the accepted answer I altered my code to deal with callbacks from other threads: I introduced a thread check at the beginning of my void doSomeWorkWithData() functions:
void doSomeWorkWithData(A a)
{
if( QThread::currentThread() != this->thread() )
{
QMetaObject::invokeMethod( this,"doSomeWorkWithData"
,Qt::QueuedConnection
,Q_ARG(A, a) );
return;
}
/* The actual work this function does would be below here... */
};
Some related thoughts:
I was contemplating to introduce a QMutexLocker before the if statement, but decided against it: the only part of the function that is potentially used in parallel (anything above the return; in the if statement) is--as far as I understand--thread safe.
Setting the connection type manually to Qt::QueuedConnection: technically, if I understand the documentation correctly, Qt should do the right thing and the default, Qt::AutoConnection, should end up becoming a Qt::QueuedConnection. But since would always be the case when that statement is reached, I decided to put explicitly in there to remind myself about why this is there.
putting the queuing code directly in the function and not hiding it in an interim function: I could have opted to put the call to invokeMethod in another interim function, say queueDoSomeWorkWithData()', which would be called by the callback in the listener and then usesinvokeMethodwith anQt::AutoConnection' on doSomeWorkWithData(). I decided against this as there seems no way for me to auto-code this interim function via templates (templates and the Meta system was part of the original problem), so "the user" of my code (i.e. the person who implements doSomeWorkWithData(XYZ xyz)) would have to hand type the interim function as well (as that is how the templated type names are correctly resolved). Including the check in the actual function seems to me to safe typing an extra function header, keeps the MyClass interface a little cleaner, and better reminds readers of doSomeWorkWithData() that there might be a threading issue lurking in the dark.
It is ok to call a public function on a subclass of QObject from another thread if you know for certain that the individual function will perform only thread-safe actions.
One nice thing about Qt is that it will handle foreign threads just as well as it handles QThreads. So, one option is to create a threadSafeDoSomeWorkWithData function for each doSomeWorkWithData that does nothing but QMetaMethod::invoke the non-threadsafe one.
public:
void threadSafeDoSomeWorkWithData(A a) {
QMetaMethod::invoke("doSomeWorkWithData", Q_ARG(A,a));
}
Q_INVOKABLE void doSomeWorkWithData(A a);
Alternatively, Sergey Tachenov suggests an interesting way of doing more or less the same thing in his answer here. He combines the two functions I suggested into one.
void Obj2::ping() {
if (QThread::currentThread() != this->thread()) {
// not sure how efficient it is
QMetaObject::invoke(this, "ping", Qt::QueuedConnection);
return;
}
// thread unsafe code goes here
}
As to why you see normal behaviour when not creating a GUI? Perhaps you're not doing anything else that is unsafe, aside from manipulating GUI objects. Or, perhaps they're the only place in which your thread-safety problems are obvious.
i am trying to refactor my project to improve testability, therefor i'm introducing an abstract factory.
My application collects data from different sources by using ICrawlers.
These ICrawlers use 3rd party libraries to access different sources, like e.g. twitter.
Example: My TwitterCrawler uses TweetSharp to access twitter data.
My first version strongly coupled the TweetSharp client to the Crawler. Now i abstracted the TweetSharp to a ITwitterClient and a TweetSharpTwitterClient implementation.
Next step is to introduce a ITwitterClientFactory with a DefaultTwitterClientFactory that creates TweetSharpTwitterClients. This should bring me closer to my goal (testability) because i can switch the factory to MockTwitterClientFactory that creates a MockTwitterClient, that delivers some test output.
Now, let me come to my point.
I am using MEF for dependency injection (but i'm rather new to it). What I'm doing is this:
public class TwitterCrawler : CrawlerBase, ICrawler
{
[Import]
public ITwitterClientFactory TwitterClientFactory {get; set;}
public override Process()
{
ITwitterClient twitterClient = TwitterClientFactory.MakeSingletonClient();
// do something with twitterClient
}
}
Whereas my DefaultTwitterClientFactory exports itself to MEF:
[Export(typeof(ITwitterClient))]
public class DefaultTwitterClientFactory: ITwitterClientFactory
{
// implementation of ITwitterClientFactory
// provides methods to create instances of ITwitterClient implementations
}
Now, while this works so far, my question is, how to switch the factory?
How can i create a unit test and use the MockClientFactory instead of the DefaultTwitterClientFactory?
Is my approach good at all? Is it better to manually set the factory that is to be used?
Somewhere something like
... new TwitterCrawler(mockedTwitterClientFactory)
or even
.... new TwitterCrawler(mockedTwitterClient)?
This actually only moves the problem outside of TwitterClient, but still somewhere i have to decide how to construct the ITwitterClient and what factory to use for that purpose.
Should i dive more into the mechanics of MEF (ExportProvider?)
You shouldn't need to use the composer/container in your unit tests - just wire the SUT directly with the Test Doubles.
Something like this:
var sut = new TwitterCrawler();
sut.TwitterClientFactory = new FakeTwitterClientFactory();
However, you should really refactor from Property Injection to Constructor Injection, as the property implies that the dependency is optional.
BTW, your DefaultTwitterClientFactory doesn't export itself, it exports ITwitterClient.