Related
I want to write a video editing software, and the "logical" conclusion is that the language I must to use is C++... But I don't like it (sorry c++ fans)
I would like to write it with something cool, like Lisp or Haskell or Erlang... But I don't know if the open source implementation of those languages (I don't have money to buy licenses) let me made a competitive software (in the performance area)
What do you think? what do you recommend?
I can't speak to Lisp, but both Erlang and Haskell are capable of the performance necessary for video processing. Achieving that performance is likely to be more difficult than with C++ because there are fewer existing libraries in the domain, so you'll have to implement more yourself. Which means you'll have to be capable of writing high-performance code yourself. In Haskell I expect this would require a significant investment of time (6 months minimum) to become proficient.
Which language you choose should depend a great deal upon the goals of the project. If it's a hobby project, or you want to learn a lot about processing algorithms (and therefore don't mind having to do a lot of low-level coding yourself), there's nothing wrong with using an out-of-mainstream language. Haskell has bindings to a lot of things you would probably want to use eventually, such as a wrapper for GLSL.
As somebody working with audio processing (including real-time), I can say that Haskell's performance hasn't been a problem for me. For a recent project I did write some functions in C, but that was necessary to implement a custom vectorization scheme. Doing high-level work in Haskell and calling out to C when necessary is a perfectly valid approach, although thankfully it's less necessary now than in the past.
Of course, this presumes a few things about the nature of your project. If you want something you can use right away, Haskell, Lisp, and Erlang are probably not the languages for you because there are fewer resources. Have you considered Processing? It's Java, I don't know if you consider that better than C++ or worse.
I had motivations besides productivity for working in Haskell (and my productivity took a big hit for a while), without those other goals I wouldn't have persevered. If you want to write something to use it, stick with what's going to be most productive. If you have other motivations, tell us what they are and it's more likely people will make helpful suggestions.
For what it's worth, Wings3D is written in Erlang.
You could always try D, if you want something somewhat similar to C++ but not C++. Also, D could use some love.
For both Haskell and Erlang, the open source implementation is the standard, most efficient available implementation available. There's no reason that Haskell shouldn't be performant enough for your needs -- for video stuff I assume you'll be using matrices and such. There are quality bindings available for BLAS & co for Haskell. I don't know of a great deal of existing video editing work, but Alberto Ruiz (the author of HMatrix) has done work with Haskell and computer vision: http://dis.um.es/profesores/alberto/research.html
There's also a great deal of work on sound libraries and processing in Haskell.
I'd use the language that gives me the best coverage by third-party libraries for what I'm trying to do; for manipulating video data that's probably going to be a mainstream language like C++.
If this project is for fun/to learn a new language then by all means, take the road less traveled. But if this is something you need to ship in a reasonable amount of time, avoiding the best tools for the job because you don't like them is unsound strategy.
That depends at least on what's your goal with the project. If it's a hobby project and you want to learn a different language, then you should choose that language. In this case, however, I assume you're familiar with video processing. On the other hand, if you want to learn about video processing, I'd recommend using a language you are already comfortable with.
Now, if it's a professional project of a decent size (video processing software can be huge) you should probably consider using different languages for different things. The kind of systems I work with usually require writing some code in C (for efficiency reasons), but we always try to keep that to de indispensable minimum and use a higher level language for most of the system behaviour (we use erlang, but that applies to any other higher level language).
IMO, writing big systems in C or C++ is almost a suicide. There are projects that succeed, but I find that much harder than complementing the C part with higher level languages.
There is already some video streaming server written in Erlang http://erlyvideo.org/. You can look for some inspiration https://github.com/erlyvideo/erlyvideo.
I am working on a new language, targeted for web development, embeding into applications, distributed applications, high-reliability software (but this is for distant future).
Also, it's target to reduce development expenses in long term - more time to write safer code and less support later. And finally, it enforces many things that real teams have to enforce - like one crossplatform IDE, one codestyle, one web framework.
In short, the key syntax/language features are:
Open source, non-restrictive licensing. Surely crossplatform.
Tastes like C++ but simpler, Pythonic syntax with strict & static type checking. Easier to learn, no multiple inheritance and other things which nobody know anyway :-)
LLVM bytecode/compilation backend gives near-C speed.
Is has both garbage collection & explicit object destruction.
Real OS threads, native support of multicore computers. Multithreading is part of language, not a library.
Types have the same width on any platform. int(32), long(64) e.t.c
Built in post and preconditions, asserts, tiny unit tests. You write a method - you can write all these things in 1 place, so you have related things in one place. If you worry that your class sourcecode will be bloated with this - it's IDEs work to hide what you don't need now.
Java-like exception handling (i.e. you have to handle all exceptions)
I guess I'll leave web & cluster features for now...
What you think? Are there any existing similar languages which I missed?
To summarize: You language has no real selling points. It just does what a dozen other languages already did, with syntax and semantics just slightly off, depending on where the programmer comes from. This may be a good thing, as it makes the language easier to adapt, but you also have to convince people to trouble to switch. All this stuff has to be built and debugged and documented again, tools have to be programmed, people have to learn it and convince their pointy-haired bosses to use it, etc. "So it's language X with a few features from Y and nicer syntax? But it won't make my application's code 15% shorter and cleaner, it won't free me from boilerplate X, etc - and it won't work with my IDE." The last one is important. Tools matter. If there are no good tools for a language, few people will shy away, rightfully so.
And finally, it enforces many things that real teams have to enforce - like one crossplatform IDE, one codestyle, one web framework.
Sounds like a downside! How does the language "enforce one X"? How do you convince programmers this coding style is the one true style? Why shouldn't somebody go and replace the dog slow, hardly maintained, severly limited IDE you "enforce" with something better? How could one web framework possibly fit all applications? Programmers rarely like to be forced into X, and they are sometimes right.
Also, you language will have to talk to others. So you have ready-made standard solutions for multithreading and web development in mind? Maybe you should start with a FFI instead. Python can use extensions written in C or C++, use dynamic libraries through ctypes, and with Cython it's amazingly simple to wrap any given C library with a Python interface. Do you have any idea how many important libraries are written in C? Unless your language can use these, people can hardly get (real-world) stuff done with it. Just think of GUI. Most mayor GUI toolkits are C or C++. And Java has hundreds of libraries (the other JVM languages profit much from Java interop) for many many purposes.
Finally, on performance: LLVM can give you native code generation, which is a huge plus (performance-wise, but also because the result is standalone), but the LLVM optimizers are limited, too. Don't expect it to beat C. Especially not hand-tuned C compiled via icc on Intel CPUs ;)
"Are there any existing similar
languages which I missed?"
D? Compared to your features:
The compiler has a dual license - GPL and Artistic
See example code here.
LDC targets LLVM. Support for D version 2 is under development.
Built-in garbage collection or explicit memory management.
core.thread
Types
Unit tests / Pre and Post Contracts
try/catch/finally exception handling plus scope guarantees
Responding to a few of your points individually (I've omitted what I consider either unimportant or good):
targeted for web development
Most people use php. Not because it's the best language available, that's for sure.
embeding into applications
Lua.
distributed applications, high-reliability software (but this is for distant future).
Have you carefully studied Erlang, both its design and its reference implementation?
it enforces many things that real teams have to enforce - like one crossplatform IDE, one codestyle, one web framework.
If your language becomes successful, people will make other IDEs, other code styles, other web frameworks.
Multithreading is part of language, not a library.
Really good languages for multithreading forbid side effects inside threads. Yes, in practice that pretty much means Erlang only.
Types have the same width on any platform. int(32), long(64) e.t.c
Sigh... There's only one reasonable width for integers outside of machine-level languages like C: infinite.
Designing your own language will undoubtedly teach you someting. But designing a good language is like designing a good cryptosystem: lots of amateurs try, but it takes an expert to do it well.
I suggest you read some of Norman Ramsey's answers here on programming language design, starting with this thread.
Given your interest in distributed applications, knowing Erlang is a must. As for sequential programming, the minimum is one imperative language and one functional language (ideally both Lisp/Scheme and Haskell, but F# is a good start). I also recommend knowing at least one high-level language that doesn't have objects, just so you understand that not having objects can often make the programmer's life easier (because objects are complex).
As for what could drive other people to learn your language... Good tools/libraries/frameworks can't hurt (FORTRAN, php), and a big company setting the example can't hurt (Java, C#). Good design doesn't seem to be much of a factor (a ha-ha-only-serious joke has it that what makes a language successful is using {braces} to delimit blocks: C, C++, Java, C#, php)...
What you've given us is a list of features, with no coherent philosophy, or explanation as to how they will work together. None of the features are unique. At best, you're offering incremental improvements over what's already there. I'd expect there's already languages kicking around with what you've said, it's just that they're still fairly obscure, because they didn't make it.
Languages have inertia. People have to learn new languages, and sometimes new tools. They need incentive to do so, and 20% improvement in a few features doesn't cut it.
What you need, at a minimum, is a killer app and a form of elevator pitch. (The "elevator pitch" is what you tell the higher-ups about your project when you're in the elevator with them, in current US business parlance.) You need to have your language be obviously worth learning for some purpose, and you need to be able to tell people why it's worth learning before they think "just another language by somebody who wanted to write a language" and go away.
You need to form a language community. That community needs to have some localization at first: people who work in X big company, people who want to do Y, whatever. Decide on what that community is likely to be, and come up with one big reason to switch and some reasons to believe that your language can deliver what it promises.
No.
Every buzzword you have included in your feature list is an enormous amount of work to be spec'd, implemented, documented, and tested.
How many people will be actively developing the language? I guess the web is full of failed programming language projects. (Same is true for non-mainstream OSes)
Have a look at what .Net/Visual Studio or Java/Eclipse have accomplished. That's 1000s of years of specification, development, tests, documentation, feedback, bug fixes, service packs.
During my last job I heard about somebody who wrote his own programming framework, because it was "better". The resulting program code (both in the framework and in the applications) is certainly unmaintainable once the original programmer quits, or is "hit by a bus", as the saying goes.
As the list sounds like Java++ or Mono++, you'd probably be more successful in engaging in an existing project, even if it won't have your name tag on it.
Perhaps you missed one key term. Performance.
In any case, unless this new language has some really out-of-this-world features(ex: 100% increase in performance over other web development languages), I think it will be yet another fish in the pond.
Currently I'm responsible for maintaining a framework developed/owned by my company. It's a nightmare. Unless there is a mainstream community, working on this full time, it's really an elephant. I do not appreciate my company's decision to develop its own framework(because it's supposed to be "faster") day 'n night.
The language tastes good in my opinion, I don't want use java for a simple website but I would like to have types and things like that. ASP .NET is a problem because of licensing and I can't afford those licenses for a single website... Also features looks good
Remember a lot of operator overloading: I think is the biggest thing that PHP is actually missing. It allows classes to behave much more like basic types :)
When you have something to test I'll love to help you with it! Thanks
Well, if you have to reinvent the wheel, you can go for it :)
I am not going to give you any examples of languages or language features, but I will give you one advice instead:
Supporting framework is what is the most important thing. People will tend to love it or hate it, depending on how easy is to write good code that get job done. Therefore, please do usability test before releasing it. I mean ask several people how they will do certain task and create API accordingly. Then test beta API on other coders and listen carefully to their comments.
Regards and good luck :)
There's always space for another programming language. Apart from getting the design right, I think the biggest problem is coming across as just another wannabe language. So you may want to look at your marketing, you need a big sponsor who can integrate your language into their products, or you need to generate a buzz around it, easiest way is astroturfing. Good luck.
http://en.wikipedia.org/wiki/List_of_programming_languages
NB the names G and G++ aren't taken. Oh and watch out for the patent trolls.
Edit
Oops G / G++ are taken... still there are plenty more letters left.
This sounds more like a "systems" language rather than a "web development language". The major languages in this category (other than C++/C) are D and Go.
My advice to you would be to not start from scratch but examine the possibility of creating tools or libraries for those languages, and seeing just how far you can push them.
Using the right language for the job is the key - this is the comment I read in SO and I also belive thats the right thing to do. Because of this we ended up using different languages for different parts of the project - like perl, VBA(Excel Macros), C# etc. We have three to four languages currently in use inside the project. Using the right language for the job has made it immensly more easy to do automate a job, but of late people are complaining that any new person who has to take over the project will have to learn so many different languages to get started. Also it is difficult to find such kind of person. Please note that this is a one to two person working on the project maximum at a given point of time. I would like to know if the method we are following is right or should we converge to single language and try to use it across all the job even though another language might be better suited for it. Your experenece related to this would also help.
Languages used and their purpose:
Perl - Processing large text file(log files)
C# with Silverlight for web based reporting.
LabVIEW for automation
Excel macros for processing data in excel sheets, generating graphs and exporting to powerpoint.
I'd say you are doing it right. Almost all the projects I've ever worked on have used multiple languages, without any problems. I think you may be overestimating the difficulty people have picking up new languages, particularly if they all use the same paradigm. If your project were using Haskell, Smalltalk, C++ and assembler, you might have difficulties, I must admit :-)
Using a variety languages vs maintainability is simply another design decision with cost-vs-benefits trade-offs that, like any design decision, need to considered carefully. While I like using the "absolute best" tool for the task (whatever "absolute best" means), I wouldn't necessarily use it without considering other factors such as:
do we have sufficient skill and experience to implement successfully
will we be able to find the necessary resources to maintain it
do we already use the language/tech in our system
does the increase in complexity to the overall system (e.g. integration issues, the impact on build automation) outweigh the benefits of using the "absolute best" language
is there another language that we already use and have experience in that is usable in lieu of the "absolute best" language
I worked a system with around a dozen engineers that used C++, Java, SQL, TCL, C, shell scripts, and just a touch of Perl. I was proud that we used the "best language" where they made sense, but, in one case, using the "best language" (TCL) was a mistake - not because it was TCL - but rather because we failed to observe the full costs-vs-benefits of the choice:*
we had only 1 engineer deeply familiar with TCL - the original engineer who refused to use anything but TCL for a particular target component - and then that engineer left the project
this target component was the only part of the system to use TCL and it was small relative to the other components in the system
the component could have been also implemented in another language we already used that we had plenty of experience in (C or C++) with some extra effort
the component appeared deceptively simple, but in reality had subtle corner cases that bit us in production (not something we could have known then, but always something to consider as a possibility later)
we had to implement special changes to the nightly build because the TCL compiler (yes, we compiled the TCL code into an executable) wouldn't work unless it could first throw its logo up on the screen - a screen which wasn't available during a cron-initiated automated nightly build. (We resorted to using xvfb to satisfy it.)
when bugs turned up, we had difficult time finding engineers who wanted to work on it
when problems with this component cropped up only after sustained load in the field, we lacked the experience and deep understanding of the TCL execution engine to easily debug it in the field
finally, the maintenance and sustainment team, which is a much smaller team with fewer resources than the main development team, had one more language that they needed training and experience in just to support this relatively small component
Although there were a number of things we could have done to head-off some of the issues we hit down the road (e.g. put more emphasis on getting TCL experience earlier, running better tests to detect the issues earlier, etc), my point is that the overall cost-vs-benefit didn't justify using TCL to code that single component. Again, it was not a mistake to use TCL because it was TCL (TCL is a fine language), but rather it was a mistake because we failed to give full consideration to the cost-vs-benefits.
As a Software Engineer it's your job to learn new languages if you need to. I would say you should go with the right tool for the job.
It's like sweeping the floor with an octopus. Yeah, it gets the job done... kind of... but it's probably not the best tool for the job. You're better off using a mop.
Another option is to create positions geared towards working in specific languages. So you can have a C# developer, a Perl developer, and a VBA developer who will only work with that language. It's a bit more overhead, but it is a workable solution.
Any modern software project of any scope -- even if it's a one-person job -- requires more than one language. For instance, a web project usually requires Javascript, a backend language, and a DB query language (though any of these might be created by the backend language). That said, there's a threshold that is easily reached, and then it would be very hard to find new developers to take over projects. What's the limit? Three languages? Four? Let's say: five is too many, but one would be too few for any reasonably complex project.
Using the right language for the right job is definitely appropriate - I am mainly a web programmer, and I need to know server-side programming (Rails, PHP + others), SQL, Javascript, jQuery, HTML & CSS (not programming languages strictly, but complex things I need to know) - it would be impossible for me to do all of that in a single language or technology.
If you have a team of smart developers, picking up new languages will not be a problem for them. In fact they probably will be eager to do so. Just make sure they are given adequate time (and mentoring) to learn the new language.
Sure, there will be a learning curve to implementing production code if there is a new language to learn, but you will have a stronger team member for it.
If you have developers in your teams who strongly resist learning new languages, unless there is a very good reason (e.g they are justifiably adamant that they are being asked to used a different language when it is not appropriate to do so) - then they are not the sort of developers I'd want in my teams.
And don't bother trying to hire people who know all the languages you use. Hire smart programmers (who probably know at least one language you use) - they should pick up the other languages just fine.
I would be of the opinion, that if I a programmer on my team wanted to introduce a second (or third) language into a project, that there better be VERY VERY good reason to do so; as a project manager I would need to be convinced that the cost of doing so, more than offset the problems. And it would take a lot of convincing.
Splitting up project into multiple languages makes it very expensive to hire the right person(s) to take over that project when it needs maintenance. For small and medium shops it could be a huge obstacle.
Edit: I am not talking about using javascript and c# on the same project, I am talking about using C# for most of the code, F# for a few parts and then VB or C++ for others - there would need to be a compelling reason.
KISS: 'Keep it simple stupid' is a good axiom to follow in most cases.
EDIT: I am not completely opposed to adding languages, but the burden of proof is on the person to who wants to do it. KISS (to me) applies to not only getting the project/product done and shipped, but also must take into account the lifetime maint. and support requirements. Lots of languages come and go, and programmers are attracted to new languages like a moth to a light. Most projects I have worked on, I still oversee and/or support 5 or 10 years later - last thing I want to see is some long forgotten and/or orphaned language responsible for some key part of an application I need to support.
My experience using C++ and Lua was that I wrote more glue than actual operational code and for dubious benefit.
I'd start by saying the issue is whether you are using the right paradigm for the job?
Suppose you know how to do object oriented programming in C#. I don't think the leap to Java is all that great . Although you'd have to familiarize yourself with libraries and syntax, the idea is pretty similar.
If you have procedural parts to your project, such as parsing files and various data transformations, your Perl/Excel Macros seem pretty similar.
But to address your issues, I'd say above all, you'll need clarity in code. Since your staff are using several languages, they won't be familiar with all languages to an equal degree. So make sure of this:
1) Syntactic sugar is explained in comments. Sugars are pretty language specific and may not be obvious to a new reader. For instance, in VBA I seem to remember there are default properties, so that TextBox1 = "Hello" is what you'd write instead of TextBox1.Text = "Hello". This can be confusing. Also, things like LINQ statements can have un-obvious meanings. So make sure people have comments to read.
2) Where two components from different languages have to work together, make excruciatingly specific details about how that happens. For instance, I once had to write a C component to be called from Excel VBA. There's quote a few steps and potential errors in doing this, especially as far as compiler flags. Make sure both sides know how their side of the interaction occurs.
Finally, as far as hiring people goes, I think you have to find people who aren't married to a specific language. To put it vaguely, hire an intelligent person who sees business issues, not code. He'll learn the lingo soon enough.
Let's say a tester is to do some programming to create automated tests ... is Lua really easy to learn for someone who is not a developer?
It depends on the particular non-developer in question. Some people will utterly block on any programming language at all. Some will easily grok many languages and basic programming concepts. There is no silver bullet for putting the power of programming in the hands of someone who is untested on it.
That being said, my personal feeling is that Lua is as good of a place to start as any other programming language.
The Lua language has an active and usually novice-friendly community. It has a long history of use on the boundary between non-programmers and programmers. The language reference manual and standard text book are among the best written examples I've seen in my career. The full text of the reference manual is online, and the first edition of Programming in Lua is as well, although the second edition of PiL reflects the differences in the language that happened after PiL was first published and is well worth the investment to purchase.
One of Lua's strengths is the ease with which it can be integrated into an existing system to construct a configuration and scripting interface to an application. That makes the development cost to adopt it relatively low. Its small size makes the impact on an application release remarkably low as well. Thus getting an existing system to the point where it can be scripted enough with Lua to use Lua as a basis for testing will likely be a straightforward task with few hidden obstacles.
Lua is very forgiving which many people associate with "easy". You do not have to enter semi-colons, you do not have to scope variables, you can write all of your functions in the global scope. Of course doing these things only make your life easier when writing. When debugging even a new programmer may soon see why taking these short cuts is not such a good idea.
I also believe that you can write very simple, easy to use APIs in Lua and you could also create very complex APIs, which may involve object oriented concepts (such as the difference between . and :) or functional APIs with closures and passing around functions as function arguments, etc. Whether the user is able to properly use and understand the language to do the task at hand depends largely on the API as much as or more so than the language.
I do believe Lua is an easier language to learn than many others, like Ruby and Python (and obviously Perl). Lua's grammar and syntax are more consistent than Ruby's for instance; in Ruby you have so many reserved keywords, plus all sorts of symbols (curly-brackets for blocks and pipes for local variables etc), plus it gives you too many options (you can either use curly-brackets for blocks, or you can use the keywords do and end to start and end blocks).
I believe that for non-programmers Lua is much easier especially because of the reasons outlined above. As for programmers, I've read many people say this very same thing and I agree: programming in Lua is very pleasant. I believe that's also because of what I said above.
It probably is becausee its very similar to Python:
The number of universities using Python in there introductory Comp Sci courses is probably the highest of any language (empirically through google). Second probably being Java and Scheme.
The number of Python libraries is astronomical. And the number of people that know the language is quite high thus if you hire a new person there is a good chance they have seen the language before.
Ironically I have grown to not like the language so I am not saying this because I am python fan boy.
As long as you clearly explain to the testers the pitfalls that they may face when debugging in LUA it shouldn't be harder than learning the programming basics of any other language.
What goes through my mind is the situation where the tester made a typo and wrote a different, yet almost unnoticeable, name for a variable. The new variable will be created with the given value but the old variable won't be modified. That sort of thing can be pretty hard to debug when people are not extremely aware of it.
I'm a programmer for more than ten years.
I've learned and used different programming languages.
I've heard about Lua in different occasions, but I've never used it before.
Recently, I decided to learn Lua because our customers are using it.
After spending days on reading the PiL, it turned out for me that Lua is a powerful, flexible, yet sophisticated programming language.
From a software developer point of view, I don't think it's easy for me to be a good coder in Lua in a short time.
But if you just want to 'do something' with Lua, especially if you are from a background of non-programmer, you may feel pleased with Lua, which may be much easier to write some ready-to-use code than some 'traditional' language such as Java, C/C++, Python, etc.
With "Polyglot" programming techniques becoming more relevant, it is almost a necessity to use the "right" PL for the problem. However, learning new languages takes time which usually most project team can't afford. What is the best way to learn a new programming language? Is there a common set of problems that can be solved to reach a certain level of competence?
Well, it depends what you want to do. (web, db, whatever).
Generally I'd want to know:
What's the library like, how do I reference it
What ORMs are there
What build/deployment platforms exist for it
How does it handle updates
How do I do general things, like:
DB Access
File things
Display UI's
and so on.
Really, learning is only by doing -- you need a project that you can use the given language for.
Project Euler is the first thing to come to mind as an oft-used set of problems to try in a new language, even if it's not something I've ever tried.
If the language is another JVM or CLR hosted one, the issues about learning the environment can be set aside -- you can use all your familiar APIs in your Clojure/Scala/F#... code -- and concentrate on the syntax and idiom.
Otherwise, you're probably using the new language because it has a good fit for the particular problem you want to solve (e.g. native code and functional -> Haskell; distributed and concurrent -> Erlang) so the fit of the feature set is known in advance but you have the extra load of learning the standard APIs. And that's what prototyping is for.
The book Programming Challenges and the associated website provide a large list of algorithmic problems, with automatic online judging in several languages (Java, C, C++). Any algorithm textbook can give you lots of examples of basic data structures and procedures to try and implement, which is often a nice way to get some practice with basic language syntax and features. My personal favourite for this is The Algorithm Design Manual, which is language agnostic, but there are plenty of good language-specific books available as well (Mastering Algorithms in Perl or Data Structures and Algorithms in Java, for example).
If you're interested in a general set of mathematical problems to try and solve, Project Euler is a great resource.
For more day to day problems, I find the cookbook approach most helpful. For example, both Perl and Python have excellent O'Reilly cookbooks, as well as online resources, which provide short examples of many common and important problems. As mentioned in another answer, the key here is to find canonical examples of basic features you will need, particularly by leveraging what's available in standard libraries. I usually try and build up my own small library of examples as I go along, e.g. a socket example, a DB access example, a file reading example, a simple numerical solver, etc, which I then pillage for ideas when it's time to write production code.