i.e. is it possible to embed Haskell code in a C library so that the user of the library doesn't have to know Haskell is being used? In particular, so that the user could use multiple libraries that embed Haskell, without any conflicts?
As far as I understand things, you embed between calls to hs_init and hs_exit, but these involve global state shenanigans and should conflict with other calls, no?
Yes, it's possible to call Haskell code from C (and vice versa) through FFI, the Foreign Function Interface. Unfortunately, as the haskell.org docs says, you can't avoid the calls to initialize and finalize the haskell environment:
The call to hs_init() initializes GHC's runtime system. Do NOT try to
invoke any Haskell functions before calling hs_init(): bad things will
undoubtedly happen.
But, this is interesting also:
There can be multiple calls to hs_init(), but each one should be
matched by one (and only one) call to hs_exit()
And furthermore:
The FFI spec requires the implementation to support re-initialising
itself after being shut down with hs_exit(), but GHC does not
currently support that.
Basically my idea is that you may exploit this specifications in order to write youself a wrapper C++ class that manages the calls to hs_init and hs_exit for you, in example by using template methods surrounded by hs_init and hs_exit that you can override using any haskell call you want.
However, beware of interactions with other libraries calling haskell code: nested layers of calls to hs_init and hs_exit should be OK (so it's safe to use libraries which calls them in between your wrappers), but the total number of calls should always match, meaning that if those libraries only initialize the environment without trying to close it, then it's up to you to finish the job.
Another (probably better) idea, without exploiting inheritance and overriding, may be to have a simple class HaskellEnv that calls hs_init in the constructor and hs_exit in the destructor. If you declare them as automatic variables, you'll obtain that the calls to hs_init and hs_exit will always be matched, and the latest call to hs_exit will be made as soon as the latest HaskellEnv object is destructed when you leave its scope.
Have a look at this question in order to prevent the creation of objects on the heap (they may be dangerous in this case).
Related
I'm writing a numerical optimisation library in Haskell, with the aim of making functions like a gradient descent algorithm available for users of the library. In writing these relatively complex functions, I write intermediary functions, such as a function that performs just one step of gradient descent. Some of these intermediary functions perform tasks that no user of the library could ever have need for. Some are even quite cryptic, but make sense when used by a bigger function.
Is it common practice to leave these intermediary functions available to library users? I have considered moving these to an "Internal" library, but moving small functions into a whole different library from the main functions using them seems like a bad idea for code legibility. I'd also quite like to test these smaller functions as well as the main functions for debugging purposes down the line - and ideally would like to test both in the same place, so that complicates things even more.
I'm unsurprisingly using Cabal for the library so answers in that context as well would be helpful if that's easier.
You should definitely not just throw such internal functions in the export of your package's trunk module, together with the high-level ones. It makes the interface/haddocks hard to understand, and also poses problems if users come to depend on low-level details that may easily change in future releases.
So I would keep these functions in an “internal” module, which the “public” module imports but only re-exports those that are indended to be used:
Public
module Numeric.Hegash.Optimization (optimize) where
import Numeric.Hegash.Optimization.Internal
Private
module Numeric.Hegash.Optimization.Internal where
gradientDesc :: ...
gradientDesc = ...
optimize :: ...
optimize = ... gradientDesc ...
A more debatable matter is whether you should still allow users to load the Internal module, i.e. whether you should put it in the exposed-modules or other-modules section of your .cabal file. IMO it's best to err on the “exposed” side, because there could always be valid use cases that you didn't foresee. It also makes testing easier. Just ensure you clearly document that the module is unstable. Only functions that are so deeply in the implementation details that they are basically impossible to use outside of the module should not be exposed at all.
You can selectively export functions from a module by listing them in the header. For example, if you have functions gradient and gradient1 and only want to export the former, you can write:
module Gradient (gradient) where
You can also incorporate the intermediary functions into their parent functions using where to limit the scope to just the parent function. This will also prevent the inner function from being exported:
gradient ... =
...
where
gradient1 ... = ...
I'd like to use Haskell's quickcheck library test some C code. The easiest way seems to be doing a foreign import and write a property on top of the resulting haskell function. The problem with this is that if the C code causes a segfault or manages to corrupt memory, my tests either crash without output or do something totally unpredictable.
The second alternative is to make simple executable wrappers over the C-bits and execute them outside the testing process via System.Process. Needless to say, doing this requires a lot of scaffolding and serializing values, but on the other hand, it can handle segfaults.
Is there any way of making the foreign import strategy as safe as running an external process?
You could implement the wrapper in your current process, but then use System.Posix.Process.forkProcess to run in safely in a process of its own, implementing the necessary communication using Haskell.
I have seen the fastcall notation appended before many functions. Why it is used?
That notation before the function is called the "calling convention." It specifies how (at a low level) the compiler will pass input parameters to the function and retrieve its results once it's been executed.
There are many different calling conventions, the most popular being stdcall and cdecl.
You might think there's only one way of doing it, but in reality, there are dozens of ways you could call a function and pass variables in and out. You could place the input parameters on a stack (push, push, push to call; pop, pop, pop to read input parameters). Or perhaps you would rather stick them in registers (this is fastcall - it tries to fit some of the input params in registers for speed).
But then what about the order? Do you push them from left to right or right to left? What about the result - there's always only one (assuming no reference parameters), so do you place the result on the stack, in a register, at a certain memory address?
Also, let's assume you're using the stack for communication - who's job is it to actually clear the stack after the function is called - the caller or the callee?
What about backing up and then restoring the contents of (certain) CPU registers - should the caller do it, or will the callee guarantee that it'll return everything the way it was?
The most popular calling convention (by far) is cdecl, which is the standard calling convention in both C and C++. The WIN32 API uses stdcall, which means any code that calls the WIN32 API needs to use stdcall for those function calls (making it another popular choice).
fastcall is a bit of an oddball - people realized for many functions with only one in/out parameter, pushing and popping from a memory-based stack is quite a bit of overhead and makes function calls a little bit heavy so the different compilers introduced (different) calling conventions that will place one or more parameters in registers before placing the rest in the stack for better performance. The problem is, not all compilers used the same rules for what goes where and who does what with fastcall, and as a result you have to be careful when using it because you'll never know who does what. Finally, see Is fastcall really faster? for info on fastcall performance benefits.
Complicated stuff.
Something important to keep in mind: don't add or change calling conventions if you don't know exactly what you're doing, because if both the caller and the callee do not agree on the calling convention, you'll likely end up with stack corruption and a segfault. This usually happens when you have the function being called in a DLL/shared library and a program is written that depends on the DLL/SO/dylib being a certain calling convention (say, cdecl), then the library is recompiled with a different calling convention (say, fastcall). Now the old program can no longer communicate with the new library.
Wikipedia states that
Conventions entitled fastcall have not been standardized, and have been implemented differently, depending on the compiler vendor. Typically fastcall calling conventions pass one or more arguments in registers which reduces the number of memory accesses required for the call.
Vtables are ubiquitous in most OO implementations, but do they have alternatives? The wiki page for vtables has a short blurb, but not really to much info (and stubbed links).
Do you know of some language implementation which does not use vtables?
Are there are free online pages which discuss the alternatives?
Yes, there are many alternatives!
Vtables are only possible when two conditions hold.
All method calls can be determined statically. If you can call functions by string name, or if you have no type information about what objects you are calling methods on, you can't use vtables because you can't map each method to the index in some table. Similarly, if you can add functions to a class at runtime, you can't assign all methods an index in the vtable statically.
Inheritance can be determined statically. If you use prototypal inheritance, or another inheritance scheme where you can't tell statically what the inheritance structure looks like, you can't precompute the index of each method in the table or what particular class's method goes in a slot.
Commonly, inheritance is implemented by having a string-based table mapping names of functions to their implementations, along with pointers allowing each class to look up its base class. Method dispatch is then implemented by walking this structure looking for the lowest class at or above the class of the receiver object that implements the method. To speed up execution, techniques like inline caching are often used, where call sites store a guess of which method should be invoked based on the type of the object to avoid spending time traversing this whole structure. The Self programming language used this idea, which was then incorporates into the HotSpot JVM to handle interfaces (standard inheritance still uses vtables).
Another option is to use tracing, where the compiler emits code that guesses what the type of the object is and then hardcodes the method to call into the trace. Mozilla Firefox uses this in its JavaScript interpreter, since there isn't a way to build vtables for every object.
I just finished teaching a compilers course and one of my lectures was on implementations of objects in various programming languages and the associated tradeoffs. If you'd like, you can check out the slides here.
Hope this helps!
I have an upcoming project in which a core requirement will be to mutate the way a method works at runtime. Note that I'm not talking about a higher level OO concept like "shadow one method with another", although the practical effect would be similar.
The key properties I'm after are:
I must be able to modify the method in such a way that I can add new expressions, remove existing expressions, or modify any of the expressions that take place in it.
After modifying the method, subsequent calls to that method would invoke the new sequence of operations. (Or, if the language binds methods rather than evaluating every single time, provide me a way to unbind/rebind the new method.)
Ideally, I would like to manipulate the atomic units of the language (e.g., "invoke method foo on object bar") and not the assembly directly (e.g. "pop these three parameters onto the stack"). In other words, I'd like to be able to have high confidence that the operations I construct are semantically meaningful in the language. But I'll take what I can get.
If you're not sure if a candidate language meets these criteria, here's a simple litmus test:
Can you write another method called clean which:
accepts a method m as input
returns another method m2 that performs the same operations as m
such that m2 is identical to m, but doesn't contain any calls to the print-to-standard-out method in your language (puts, System.Console.WriteLn, println, etc.)?
I'd like to do some preliminary research now and figure out what the strongest candidates are. Having a large, active community is as important to me as the practicality of implementing what I want to do. I am aware that there may be some unforged territory here, since manipulating bytecode directly is not typically an operation that needs to be exposed.
What are the choices available to me? If possible, can you provide a toy example in one or more of the languages that you recommend, or point me to a recent example?
Update: The reason I'm after this is that I'd like to write a program which is capable of modifying itself at runtime in response to new information. This modification goes beyond mere parameters or configurable data, but full-fledged, evolved changes in behavior. (No, I'm not writing a virus. ;) )
Well, you could always use .NET and the Expression libraries to build up expressions. That I think is really your best bet as you can build up representations of commands in memory and there is good library support for manipulating, traversing, etc.
Well, those languages with really strong macro support (in particular Lisps) could qualify.
But are you sure you actually need to go this deeply? I don't know what you're trying to do, but I suppose you could emulate it without actually getting too deeply into metaprogramming. Say, instead of using a method and manipulating it, use a collection of functions (with some way of sharing state, e.g. an object holding state passed to each).
I would say Groovy can do this.
For example
class Foo {
void bar() {
println "foobar"
}
}
Foo.metaClass.bar = {->
prinltn "barfoo"
}
Or a specific instance of foo without effecting other instances
fooInstance.metaClass.bar = {->
println "instance barfoo"
}
Using this approach I can modify, remove or add expression from the method and Subsequent calls will use the new method. You can do quite a lot with the Groovy metaClass.
In java, many professional framework do so using the open source ASM framework.
Here is a list of all famous java apps and libs including ASM.
A few years ago BCEL was also very much used.
There are languages/environments that allows a real runtime modification - for example, Common Lisp, Smalltalk, Forth. Use one of them if you really know what you're doing. Otherwise you can simply employ an interpreter pattern for an evolving part of your code, it is possible (and trivial) with any OO or functional language.