Related
As a platform for handling concurrent problems, Elixir/OTP seems to be the best suited solution.
When writing an application with a web interface, consider the case in which I want to reason about, and decouple, the application logic using another functional language - namely haskell (due to benefits like its advanced detection of errors at compile time, static typing, etc). I would then handle concurrency using GenServers, and attach a web interface using Phoenix.Channels.
Is this setup even possible using NIFs? Also, would true concurrency be maintained? I'm not sure that I'm following the correct line of reasoning here, but would a new haskell process be able to be spawned in line with GenServer demands, and would the two be able communicate efficiently?
This setup is certainly possible using NIFs and GHC's FFI with a small amount of boilerplate written in C. But NIFs are best used for short synchronous computations with no side effects and I get the feeling that that isn't what these operations are.
You'd probably be better off with C Nodes for the Haskell parts of the application. Most of the documentation you'll find for that will be for Erlang and not Elixir, but given Elixir's easy interop with Erlang, it should be pretty straight forward (someone's even written an example). Most of the hard work will be to do with writing a Haskell "C Node", for which a cursory glance at hackage and github turns up nothing.
This question is about design / is fairly open-ended.
I'd like to use OpenCV, a large C++ library, from Haskell.
The closest solution at the moment is probably Arjun Comar's attempt to adapt the Python / Java binding generator.
See here, here, and here.
His approach generates a C interface, which is then wrapped using hsc2hs.
Due to OpenCV's lack of referential transparency in its API, as well as its frequent use of call parameters for output, for Arjun's approach to fully succeed he'll need to define a new API for OpenCV, and implement it in terms of the existing one.
So, it seems it might not be too much extra work to go whole-hog and define an API using an interface description languages (IDL), such as SWIG, protobuf-with-RPC, or Apache Thrift.
This would provide interfaces to a number of languages besides Haskell.
My questions:
Is there anything better than SWIG for a server-free solution?
(I just want to call into C++; I'd rather not go through a local server.)
If there's no good server-free solution, should I use protobuf-with-RPC or Thrift?
Related: How good is Thrift's Haskell support?
From the code, it looks like it needs updating (I see references to GHC 6).
Related: What's a good protobuf-with-RPC solution?
With Apache Thrift, you get Haskell support. You are correct, code is not generally "latest", but you rarely care. You can do complex things on other abstraction levels and keep things as simple as possible at messaging level.
Google Protobuf has no support for Haskell, nor does SWIG. With Protobuf you get C++, Java, JavaScript and Python, to my knowledge the main languages at Google. Have a look at this presentation. Without contest, Thrift and Protobuf are the best in house.
It seems in your case you have to go with Thrift, as it supports Haskell.
It sounds like the foreign function interface for C++ is what you want:
Hackage,
Github
Disclaimer: I haven't used it, only heard good things about it.
I want to write a video editing software, and the "logical" conclusion is that the language I must to use is C++... But I don't like it (sorry c++ fans)
I would like to write it with something cool, like Lisp or Haskell or Erlang... But I don't know if the open source implementation of those languages (I don't have money to buy licenses) let me made a competitive software (in the performance area)
What do you think? what do you recommend?
I can't speak to Lisp, but both Erlang and Haskell are capable of the performance necessary for video processing. Achieving that performance is likely to be more difficult than with C++ because there are fewer existing libraries in the domain, so you'll have to implement more yourself. Which means you'll have to be capable of writing high-performance code yourself. In Haskell I expect this would require a significant investment of time (6 months minimum) to become proficient.
Which language you choose should depend a great deal upon the goals of the project. If it's a hobby project, or you want to learn a lot about processing algorithms (and therefore don't mind having to do a lot of low-level coding yourself), there's nothing wrong with using an out-of-mainstream language. Haskell has bindings to a lot of things you would probably want to use eventually, such as a wrapper for GLSL.
As somebody working with audio processing (including real-time), I can say that Haskell's performance hasn't been a problem for me. For a recent project I did write some functions in C, but that was necessary to implement a custom vectorization scheme. Doing high-level work in Haskell and calling out to C when necessary is a perfectly valid approach, although thankfully it's less necessary now than in the past.
Of course, this presumes a few things about the nature of your project. If you want something you can use right away, Haskell, Lisp, and Erlang are probably not the languages for you because there are fewer resources. Have you considered Processing? It's Java, I don't know if you consider that better than C++ or worse.
I had motivations besides productivity for working in Haskell (and my productivity took a big hit for a while), without those other goals I wouldn't have persevered. If you want to write something to use it, stick with what's going to be most productive. If you have other motivations, tell us what they are and it's more likely people will make helpful suggestions.
For what it's worth, Wings3D is written in Erlang.
You could always try D, if you want something somewhat similar to C++ but not C++. Also, D could use some love.
For both Haskell and Erlang, the open source implementation is the standard, most efficient available implementation available. There's no reason that Haskell shouldn't be performant enough for your needs -- for video stuff I assume you'll be using matrices and such. There are quality bindings available for BLAS & co for Haskell. I don't know of a great deal of existing video editing work, but Alberto Ruiz (the author of HMatrix) has done work with Haskell and computer vision: http://dis.um.es/profesores/alberto/research.html
There's also a great deal of work on sound libraries and processing in Haskell.
I'd use the language that gives me the best coverage by third-party libraries for what I'm trying to do; for manipulating video data that's probably going to be a mainstream language like C++.
If this project is for fun/to learn a new language then by all means, take the road less traveled. But if this is something you need to ship in a reasonable amount of time, avoiding the best tools for the job because you don't like them is unsound strategy.
That depends at least on what's your goal with the project. If it's a hobby project and you want to learn a different language, then you should choose that language. In this case, however, I assume you're familiar with video processing. On the other hand, if you want to learn about video processing, I'd recommend using a language you are already comfortable with.
Now, if it's a professional project of a decent size (video processing software can be huge) you should probably consider using different languages for different things. The kind of systems I work with usually require writing some code in C (for efficiency reasons), but we always try to keep that to de indispensable minimum and use a higher level language for most of the system behaviour (we use erlang, but that applies to any other higher level language).
IMO, writing big systems in C or C++ is almost a suicide. There are projects that succeed, but I find that much harder than complementing the C part with higher level languages.
There is already some video streaming server written in Erlang http://erlyvideo.org/. You can look for some inspiration https://github.com/erlyvideo/erlyvideo.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
Have you actually "tried" (means programmed in, not just read an article on it) Erlang and decided against it for a project? If so, why? Also, if you have opted to go back to your old language, or to use another functional language like F#, Haskell, Clojure, Scala, or something else then this counts too, and state why.
I returned to Haskell for my personal projects from Erlang for the simple virtue of Haskell's amazing type system. Erlang gives you a ton of tools to handle when things go wrong. Haskell gives you tools to keep you from going wrong in the first place.
When working in a language with a strong type system you are effectively proving free theorems about your code every time you compile.
You also get a bunch of overloading sugar from Haskell's typeclass machinery, but that is largely secondary to me -- even if it does allow me to express a number of abstractions that would be terribly verbose or non-idiomatic and unusable in Erlang (e.g. Haskell's category-extras).
I love Erlang, I love its channels and its effortless scalability. I turn to it when these are the things I need. Haskell isn't a panacea. I give up a better operational understanding of space consumption. I give up the magical one pass garbage collector. I give up OTP patterns and all that effortless scalability.
But its hard for me to give up the security blanket that, as is commonly said, in Haskell, if it typechecks, it is probably correct.
We use Haskell, OCaml and (now) F# so for us it has nothing to do with lack of C-like syntax. Rather we skip Erlang because:
It's dynamically typed (we're fans of Haskell's type system)
Doesn't provide a 'real' string type (I understand why, but it's annoying that this hasn't been corrected at the language level yet)
Tends to have poor (incomplete or unmaintained) database drivers
It isn't batteries included and doesn't appear to have a community working on correcting this. If it does, it isn't highly visible. Haskell at least has Hackage, and I'd guess that's what has us choosing that language over any other. In Windows environments F# is about to have the ultimate advantage here.
There are probably other reasons I can't think of right now, but these are the major points.
The best reason to avoid Erlang is when you cannot commit to the functional way of programming.
I read an anti-Erlang blog rant a few weeks ago, and one of the author's criticisms of Erlang is that he couldn't figure out how to make a function return a different value each time he called it with the same arguments. What he really hadn't figured out is that Erlang is that way on purpose. That's how Erlang manages to run so well on multiple processors without explicit locking. Purely functional programming is side-effect-free programming. You can arm-twist Erlang into working like our ranting blogger wanted, adding side effects, but in doing so you throw away the value Erlang offers.
Pure functional programming is not the only right way to program. Not everything needs to be mathematically rigorous. If you determine your application would be best written in a language that misuses the term "function", better cross Erlang off your list.
I have used Erlang in a few project already. I often use it for restful services. Where I don't use it however is for complex front end web applications where tools like Ruby on Rails are far better. But for the powerbroker behind the scenes I know of no better tool than Erlang.
I also use a few applications written in Erlang. I use CouchDB and RabbitMQ a bit and I have set up a few EJabberd servers. These applications are the most powerful, easiest and flexible tools in their field.
Not wanting to use Erlang because it does not use JVM is in my mind pretty silly. JVM is not some magical tool that is the best in doing everything in the world. In my mind the ability to choose from an arsenal of different tools and not being stuck in a single language or framework is what separates experts from code monkeys.
PS: After reading my comment back in context I noticed it looked like I was calling oxbow_lakes a code monkey. I really wasn't and apologize if he took it like that. I was generalizing about types of programmers and I would never call an individual such a negative name based on one comment by him. He is probably a good programmer even though I encourage him to not make the JVM some sort of a deal breaker.
Whilst I haven't, others on the internet have, e.g.
We investigated the relative merits of
C++ and Erlang in the implementation
of a parallel acoustic ray tracing
algorithm for the U.S. Navy. We found
a much smaller learning curve and
better debugging environment for
parallel Erlang than for
pthreads-based C++ programming. Our
C++ implementation outperformed the
Erlang program by at least 12x.
Attempts to use Erlang on the IBM Cell
BE microprocessor were frustrated by
Erlang's memory footprint. (Source)
And something closer to my heart, which I remember reading back in the aftermath of the ICFP contest:
The coding was very straightforward,
translating pseudocode into C++. I
could have used Java or C#, but I'm at
the point where programming at a high
level in C++ is just as easy, and I
wanted to retain the option of quickly
dropping down into some low-level
bit-twiddling if it came down to it.
Erlang is my other favorite language
for hacking around in, but was worried
about running into some performance
problem that I couldn't extricate
myself from. (Source)
For me, the fact that Erlang is dynamically typed is something that makes me wary. Although I do use dynamically typed languages because some of them are just so very problem-oriented (take Python, I solve a lot of problems with it), I wish they were statically typed instead.
That said, I actually intended to give Erlang a try for some time, and I’ve just started downloading the source. So your “question” achieved something after all. ;-)
I know Erlang since university, but have never used it in my own projects so far. Mainly because I'm mostly developing desktop applications, and Erlang is not a good language for making nice GUIs. But I will soon implement a server application, and I will give Erlang a try, because that's what it's good for. But I'm worring that I need more librarys, so maybe I'll try with Java instead.
A number of reasons:
Because it looks alien from anyone used to the C family of languages
Because I wanted to be able to run on the Java Virtual Machine to take advantage of tools I knew and understood (like JConsole) and the years of effort which have gone into JIT and GC.
Because I didn't want to have to rewrite all the (Java) libraries I've built up over the years.
Because I have no idea about the Erlang "ecosystem" (database access, configuration, build etc).
Basically I am familiar with Java, its platform and ecosystem and I have invested much effort into building stuff which runs on the JVM. It was easier by far to move to scala
I Decided against using Erlang for my project that was going to be run with a lot of shared data on a single multi-processor system and went with Clojure becuase Clojure really gets shared-memory-concurrency. When I have worked on distributed data storage systems Erlang was a great fit because Erlang really shines at distributed message passing systems. I compare the project to the best feature in the language and choose accordingly
Used it for a message gateway for a proprietary, multi-layered, binary protocol. OTP patterns for servers and relationships between services as well as binary pattern matching made the development process very easy. For such a use case I'd probably favor Erlang over other languages again.
The JVM is not a tool, it is a platform. Although I am all in favour of choosing the best tool for the job the platform is mostly already determined. Unless I am developing something standalone, from scratch and without the desire to reuse any existing code/library (three aspects that are unlikely in isolation already) I may be free to choose the platform.
I do use multiple tools and languages but I mainly targetg the JVM platform. That precludes Erlang for most if not all of my projects, as interesting as some of it concepts are.
Silvio
While I liked many design aspects of the Erlang runtime and the OTP platform, I found it to be a pretty annoying program language to develop in. The commas and periods are totally lame, and often require re-writing the last character of many lines of code just to change one line. Also, some operations that are simple in Ruby or Clojure are tedious in Erlang, for example string handling.
For distributed systems relying on a shared database the Mnesia system is really powerful and probably a good option, but I program in a language to learn and to have fun, and Erlang's annoying factor started to outweigh the fun factor once I had gotten past the basic bank account tutorials and started writing plugins for an XMPP server.
I love Erlang from the concurrency standpoint. Erlang really did concurrency right. I didn't end up using erlang primarily because of syntax.
I'm not a functional programmer by trade. I generally use C++, so I'm covet my ability to switch between styles (OOP, imperative, meta, etc). It felt like Erlang was forcing me to worship the sacred cow of immutable-data.
I love it's approach to concurrency, simple, beautiful, scalable, powerful. But the whole time I was programming in Erlang I kept thinking, man I'd much prefer a subset of Java that disallowed data sharing between thread and used Erlangs concurrency model. I though Java would have the best bet of restricting the language the feature set compatible with Erlang's processes and channels.
Just recently I found that the D Programing language offers Erlang style concurrency with familiar c style syntax and multi-paradigm language. I haven't tried anything massively concurrent with D yet, so I can't say if it's a perfect translation.
So professionally I use C++ but do my best to model massively concurrent applications as I would in Erlang. At some point I'd like to give D's concurrency tools a real test drive.
I am not going to even look at Erlang.
Two blog posts nailed it for me:
Erlang machinery walks the whole list to figure out whether they have a message to process, and the only way to get message means walking the whole list (I suspect that filtering messages by pid also involves walking the whole message list)
http://www.lshift.net/blog/2010/02/28/memory-matters-even-in-erlang
There are no miracles, indeed, Erlang does not provide too many services to deal with unavoidable overloads - e.g. it is still left to the application programmer to deal checking for available space in the message queue (supposedly by walking the queue to figure out the current length and I suppose there are no built-in mechanisms to ensure some fairness between senders).
erlang - how to limit message queue or emulate it?
Both (1) and (2) are way below naive on my book, and I am sure there are more software "gems" of similar nature sitting inside Erlang machinery.
So, no Erlang for me.
It seems that once you have to deal with a large system that requires high performance under overload C++ + Boost is still the only game in town.
I am going to look at D next.
I wanted to use Erlang for a project, because of it's amazing scalability with number of CPU'S. (We use other languages and occasionally hit the wall, leaving us with having to tweak the app)
The problem was that we must deliver our application on several platforms: Linux, Solaris and AIX, and unfortunately there is no Erlang install for AIX at the moment.
Being a small operation precludes the effort in porting and maintaining an AIX version of Erlang, and asking our customers to use Linux for part of our application is a no go.
I am still hoping that an AIX Erlang will arrive so we can use it.
With regards to making a game server, it seems Erlang always comes up as a language that is "built for this kind of thing" with its scalability and concurrency features. I don't have experience in either Haskell nor Erlang, but on the surface they seem the same. Looking into Haskell's docs it seems like it has support for multiprocessor scalability and concurrency, and Haskell is said to be a more solid language and has a visibly better community. My question is, then, is Haskell considered to be as good of a solution to server building as Erlang supposedly is?
It depends what you want to do with your server. As might be expected from a telecoms application, Erlang excels at doing simple tasks with very high concurrency. If your server will need a bazillion connections per second, or at once, Erlang is your friend. Erlang also offers better support for distributing load over multiple servers.
Haskell excels at complex, symbolic computation and as of April 2009 can also handle a great many threads (see update below). Moreover, Haskell has more tools for getting complicated code right: things like QuickCheck, SmallCheck, and the static type system. So if your server is doing complicated, interesting things and you can get by with just one server, you're probably better off with Haskell.
Update 13 April 2009: Don Stewart, a reliable source, reports that "the last thread-scaling bug in the Glasgow Haskell Compiler was squished a few months ago", and that some users report using a million Haskell threads without trouble. As of January 2009, there is a new, unpublished paper from the implementors which may describe how this is achieved.
Update 21 February 2012: John Hughes's company, QuviQ, now makes QuickCheck for Erlang. They have found a number of very interesting bugs. You can download "QuickCheck Mini" for free; it is comparable to the Haskell QuickCheck. There is also a more powerful commercial version.
The benchmarks of these papers show that Haskell can compete with Apache:
Developing a high-performance web server in Concurrent Haskell
— Simon Marlow
Combining Events And Threads For Scalable Network Services:
Implementation And Evaluation Of Monadic,
Application-level Concurrency Primitives
— Peng Li Stephan A. Zdancewic (see Figure 19)
I don't have experience in either Haskell nor Erlang, but on the surface they seem the same.
There are some pretty stark differences between Haskell and Erlang. Erlang is specifically designed for concurrent systems. The language and virtual machine are both designed to support many, many processes, and Erlang uses an actor-style system to manage communication between all of them. Haskell also supports concurrency fairly easily, due to its functional nature, but it's still a bit harder to do concurrent programming in Haskell, and the language isn't specifically set up to facilitate this.
Like Haskell, Erlang doesn't share state between processes, so it's easy to write multi-process software. But the programming style between Haskell and Erlang is a bit different, since Erlang emphasizes the use of small processes to perform concurrent processing.
I love Haskell -- it's one of my favorite languages -- but if I was going to write server software, I'd probably use Erlang. But it's certainly possible to write a server in Haskell, if you know Haskell better or find the library support to be superior.
Now there is a new option: use the Haskell/Erlang FFI to write your logic in Haskell and communicate using Erlang.
Last time I looked, the libraries and frameworks for building scalable servers in Erlang looked a bit more mature than those for Haskell. I'd suggest looking at Programming Erlang: Software for a Concurrent World for information on those.
It is a lot easier to introduce memory leaks in your Haskell application due to laziness. Long-running servers are precisely the kind of programs where you really don't want to have any memory leaks.
While I agree that Haskell is a more solid language and nicer to program in, Erlang is much easier and has many libraries specifically intended for uses like these.
I don't think there is a Haskell equivalent to, say, Mnesia, and writing it is going to be difficult. You can write Haskell versions of gen_server, gen_event, etc. but they won't have been optimised and tuned for over a decade.
The question of whether Haskell is as good as Erlang depends on what people want from a language. I think both would do quite well as a game server, but it depends mainly on what you want or expect from a programming language. One of the easiest differences to note is that Haskell is a statically typed language with type inference, and Erlang is a dynamically typed language. Overall, I would say that Haskell requires a bit more "sophistication" for those not accustomed to functional programming.
I'm sure you can find people who think so, but I think you're mistaken about the ability of Erlang to support this kind of use; it's widely used in telephony applications, and is actually quite robust. Erlang is very much optimized to highly reliable, high concurrency servers.