I am an autodidact, teaching myself how to program and script in different languages (novice in: Java, C++, Javascript/Node.js, HTML/CSS), design projects and schematics, adding electronics and peripherals.
What I have seen a lot of while researching is the use of multiple languages to achieve a set of goals (such as building a Web server in Javascript/Node to handle HTTP requests and responses, responding with a Web page written in HTML and customized/stylized with CSS and embedded with Javascript mannerisms; or instead of Node, you write it in PHP or Python).
I'm having a hard time wrapping my mind around WHY multiple languages are used instead of just one (some high-level languages are capable of performing a large portion, if not all, of the required tasks) and HOW information is passed in between the different languages. Could one program call another (I know that an HTML file can make "calls" to CSS and Javascript files, so, I understand that instance)?
I think the reason why I am hung up on this is because of my inexperience and lack of knowledge of other common languages. Does that mean that certain languages are meant to handle only specific tasks in a specific manner?
I feel like some languages, such as Java and C++, for example, can be used in various ways and in various instances to handle a myriad of different tasks. Is that not true of some of the others (PHP & Python, for instance)?
I'm digging into the wealth of knowledge and the collective experience of some of the most brilliant minds this world has to offer but remember that I am new to this and I don't have the advantage of doing this in a classroom but I have read and own many books on programming in specific languages and the like. Please answer in a way that I and the others that may follow can understand.
Thank you for your time and I look forward to the responses.
Cheers.
Fantastic answers!
I'm curious though; when working towards a solution for a specific problem, when does the programmer know when to stop in one language and continue a segment in another language?
That's where I am confused. Is it typically up to the software developer and his/her own particular and artistic preference on how something is done or are certain things just not possible without using multiple languages?
I do understand scripting and when it's beneficial to use rather than a program or application and I know runtime execution/compiled code, environments and frameworks and virtual machines but none of that clearly lays out a defined perimeter or a limit in functionality/ability for any particular language. Why call a C++ function in Python? Could Python not accomplish what was needed in the first place and could choosing a more appropriate language have mitigated the need for adding another level of complexity to the solution? I may be overthinking it but knowing this will guide me in my learning and help me map out better solutions as a programmer.
Basically the different technologies (browsers, operating systems etc) and with it programming languages evolved over time so that there are many different languages used in practice. For the same reason that there are multiple real languages. You could design a web browser that supports Python instead of JavaScript for front-end programming, but this would involve designing the APIs the scripts use to access the pages (DOM HTML model), it would need to be supported by all major web browsers, standardized, and web developers would need to use it.
Yes in many cases it is possible for programs written in one language to call programs written in another languages. There needs to be some kind of interface connecting the two parts, depending on the context. For example:
C and C++ are both compiled languages. That is, they get translated into machine code to be executed by the processor. The positions in the machine code where code for one function is located are stored with it. The linker of the operating system is responsible for linking two modules (.c files) such that function calls made in one module towards a function defined in the other result in the right machine code being loaded. For a C++ program to call a C program, one problem (many others) is that functions are named differently (name mangling). In practice the functions from the C program would need to be declared extern "C" in the C++ source code for the linker to set this up correctly.
JavaScript, CSS and HTML are interpreted and executed (for JavaScript) by the browser, but not necessarily translated into machine code. (JavaScript engined may use Just-in-time compilation). So the browser provides possibilities for the JavaScript code to access the CSS definition, for example. .style.color = ....
For scripting languages like Perl, PHP, Python etc to call each other, different libraries exist that handle the necessary intermediate steps ("glue code"). There are many possibilities, for instance the PHP code could invoke the Python interpreter to execute a Python program, or it could pass data to a running Python program through the operating system's mechanisms etc.
Wrappers such as SWIG allow C/C++ code to be called from scripting languages. They add the necessary symbols (functions) to the code that Python would call internally. Than the C++ program is compiled as a Python extension, which is loaded by the Python interpreter, itself a compiled program, and the operating system's linker is used. The Python interpreter then interprets Python code in such a way that calling a given Python function results in the machine code of the extension's wrapper function to be executed.
There are many ways to classify programming languages into categories. For example from low level (machine code) to higher level (more abstraction, translation to machine code handled automatically):
Assembly (for expressing machine code instructions)
Compiled languages for system-level programming. (C, C++, Pascal, ...)
Compiled languages running in a VM (Java, C#, ...)
Scripting languages (Python, Perl, PHP, ...) Less focused on efficiency, but more flexible.
Higher domain-speficic level languages (MATLAB, AppleScript)
Shell scripting (bash, sh)
Programming is all about creating solutions to problems. People think differently. People see the world from different perspectives. People like to tweak solutions and play with tools. Languages are created by people in order to solve different problems and in some cases just for play. My response is more along the lines of 'Why would there only be one language?'.
Related
I am searching for two (one?) languages. One of them would be a host, capable running some "environment", and second, which could used to script "agents" acting in this environment.
Some details:
host should be capable to run multiple "agents" (threads of embedded language), ideally capable to limit number of instructions executed by every thread at a time (though more sophisticated ways of control are welcome)
embedded threads should have access only to objects explicitly exposed by host
embedded threads should be isolated. No shared memory, all communication going via host
embedded language should be rather simple, with dynamic typing
hosting language should be rather high-level
performance is not a primary concern
I was thinking of Python being a host embedding Lua, for example Lunatic Python, or some pair of JVM languages (Scala / Groovy ?), but i'am not sure of possibilities of real isolation of embedded threads (see 2, 3). So I am searching for any ideas, frameworks, successfull implementations etc...
I think the conditions that you have listed aren't very restrictive, you are going to find a large list of pairs of host/thread scripting languages that fall within your six requirements.
So my most important recommendation is that you choose the languages that the intended users will like the most.
There are a couple of ways to approach this. If you decide to start picking a host language, then I think you have the following options:
C/C++: If your intention is to have more freedom of choice for the thread language, then having a C or C++ host will give you the most options, since most interpreted scripting languages are written in C/C++ and have easy mechanisms for embedding into C/C++ applications. Choices for thread languages could be Lua, Javascript, Python, Ruby, PHP, Basic, Scheme, Pascal, Lisp, and many many more.
Java: With Java as a host language you have a short number of scripting languages that are embed friendly. Here is a list.
.NET: I would only go with this if you intend to run on Windows and nothing else, if not I would avoid it. Like Java, you'll find a list of interpreters that can be embedded in a .NET application.
Something else: If you don't like C/C++ or Java as a host language, then you'll have to decide what kind of host language you like, but no matter the choice, you will have a very limited set of options for an embedded language. You suggested Python as a possible host language, which I think is a decent choice. I would advise against using Lua over Python though, that could be a debugging nightmare should you ever need to debug at the language VM level. Instead, I can offer two suggestions: (1) also use Python for the thread language, then you have a nice uniform language across the entire system; (2) find a scripting language that has a native interpreter written in Python (there aren't many that I know of). Instead of Python, you can go with Ruby, PHP, or any other major scripting language for the host, but in all cases you'll have not many options for the embedded language.
Now, instead of looking for a host language, you could pick the embedded language first, then find a host language that can embed that language. You suggested Lua as an embedded language, which is also a very good option. If you are set on using Lua, then I think C or C++ should be your host language, as that will give you the best embedding experience.
To summarize, I recommend one of two following approaches to choosing your pair of languages:
(a) pick a pair of languages where the embedded language interpreter is implemented in the host language. Examples: Lua and C++, Javascript and C++, Python and C++, Scheme and C++, Jython and Java, JRuby and Java, Rhino and Java, etc.
or
(b) pick the same high level scripting language for host and threads, and work on an unified platform without embedded scripting. Examples: Python and Python, Lua and Lua, Ruby and Ruby, etc.
Good luck with your search!
You could also use Common Lisp in particular thru its SBCL implementation.
Using javascript/v8 as the embedded language, and c++ as the host language might be a good solution. See this for how to embed.
V8 provides Contexts, each of which has its own Security Context. These allow you to create multiple separate threads for different clients, each of which is in their own sandbox.
Another alternative is java/javascript(rhino) which also allows embedding. Limiting access to other objects will be harder in java (you have to use a security manager), but you can limit the time a script is allowed to take, see an example in the javadocs here.
I know this question is a couple of years old now, but I'd recommend ObjectScript language.
ObjectScript, OS for short, is a new programming language. It's free, cross-platform, lightweight, embeddable and open-source. It combines the benefits of multiple languages, including: JavaScript, Lua, Ruby, Python and PHP. OS features the syntax of Javascripts, the "multiple results" feature from lua, syntactic shugar from Ruby as well as magic methods from PHP and Ruby - and even more!
A good example here is SnapScript it can be run on any Java compatible environment. In addition to the standard JRE it can also be run on Android (Dalvik and ART) without any modifications.
The IDE actually manages an agent pool for hot execution.
We all know that MetaProgramming is a Concept of Code == Data (or programs that write programs).
But are there any applications that use it & what are the advantages of using it?
This Question can be closed but i didnt see any related questions.
IDEs are full with metaprogramming:
code completion
code generation
automated refactoring
Metaprogramming is often used to work around the limitations of Java:
code generation to work around the verbosity (e.g. getter/setter)
code generation to work around the complexity (e.g. generating Swing code from a WYSIWIG editor)
compile time/load time/runtime bytecode rewriting to work around missing features (AOP, Kilim)
generating code based on annotations (Hibernate)
Frameworks are another example:
generating Models, Views, Controllers, Helpers, Testsuites in Ruby on Rails
generating Generators in Ruby on Rails (metacircular metaprogramming FTW!)
In Ruby, you pretty much cannot do anything without metaprogramming. Even simply defining a method is actually running code that generates code.
Even if you just have a simple shell script that sets up your basic project structure, that is metaprogramming.
Since code as data is one of key concepts of Lisp, the best thing would be to see the real applications of projects written in these.
On this link you can see an article about a real world application written partly in Clojure, a dialect of Lisp.
The thing is not to write programs that write programs, just because you can, but to add new functionality to your language when you really need it. Just think if you could simply add new keyword to Java or C#...
If you implement metaprogramming in a language-independent way, you get a program analysis and transformation system. This is precisely a tool that treats (arbitrary) programs as data. These can be used to carry out arbitrary transformations on arbitrary programs.
It also means you aren't limited by the specific metaprogramming features that the compiler guys happened to put into your language. For instance, while C++ has templates, it has no "reflection". But a program transformation system can provide reflection even if the base langauge doesn't have it. In particular, having a program transformation engine means never having to say "I'm sorry, your language doesn't support metaprogramming (well enough) so I can't do much except write code manually".
See our DMS Software Reengineering Toolkit for such a program transformation system. It has been used to build test coverage and profiling tools, code generation tools, tools to reshape the architecture of large scale C++ applications, tools to migrate applications from one langauge to another, ... This is all extremely practical. Most of the tasks done with DMS would completely impractical to do by hand.
Not a real world application, but a talk about metaprogramming in ruby:
http://video.google.com/videoplay?docid=1541014406319673545
Google TechTalks August 3, 2006 Jack Herrington, the author of Code Generation in Action (Manning, July 2003) , will talk about code generation techniques using Ruby. He will cover both do-it-yourself and off-the-shelf solutions in a conversation about where Ruby is as a tool, and where it's going.
A real world example would be Django's model metaclass. It is the class of the class, from which models inherit from and responsible for the outfit of the model instances with all their attributes and methods.
Any ORM in a dynamic language is an instant example of practical metaprogramming. E.g. see how SQLAlchemy or Django's ORM creates classes for tables it discovers in the database, dynamically, in runtime.
ORMs and other tools in Java world that use #annotations to modify class behavior do a bit of metaprogramming, too.
Metaprogramming in C++ allows you to write code that will get transformed at compilation.
There are a few great examples I know about (google for them):
Blitz++, a library to write efficient code for manipulating arrays
Intel Array Building Blocks
CGAL
Boost::spirit, Boost::graph
Many compilers and interpreters are implemented with metaprogramming techniques internally - as a chain of code rewriting passes.
ORMs, project templates, GUI code generation in IDEs had been mentioned already.
Domain Specific Languages are widely used, and the best way to implement them is to use metaprogramming.
Things like Autoconf are obviously cases of metaprogramming.
Actually, it's unlikely one can find an area of software development which won't benefit from one or another form of metaprogramming.
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
When is a language considered a scripting language?
I am really confused between different types of languages.
Can any one guide what are diff types of languages or diff categories.
Like some saying python is scripting langauge. Now what does that mean. Are other langueages like php , asp , java not scripting langauges
The name "Scripting language" applies to a very specific role: the language which you write commands to send to an existing software application. (like a traditional tv or movie "script")
For example, once upon a time, HTML web pages were boring. They were always static. Then one day, Netscape thought, "Hey, what if we let the browser read and act on little commands in the page?" And like that, Javascript was formed.
A simple javascript command is the alert() command, which instructs/commands the browser (a software app) that is reading the webpage to display an alert.
Now, does alert() related, in any way, to the C++ or whatever code language that the browser actually uses to display the alert? Of course not. Someone who writes "alert()" on an .html page has no understanding of how the browser actually displays the alert. He's just writing a command that the browser will interpret.
Let's see the simple javascript code
<script>
var x = 4
alert(x)
</script>
These are instructs that are sent to the browser, for the browser to interpret in itself. The programming language that the browser goes through to actually set a variable to 4, and put that in an alert...it is completely unrelated to javascript.
We call that last series of commands a "script" (which is why it is enclosed in <script> tags). Just by the definition of "script", in the traditional sense: A series of instructions and commands sent to the actors. Everyone knows that a screenplay (a movie script), for example, is a script.
The screenplay (script) is not the actors, or the camera, or the special effects. The screenplay just tells them what to do.
Now, what is a scripting language, exactly?
There are a lot of programming languages that are like different tools in a toolbox; some languages were designed specifically to be used as scripts.
Javasript is an obvious example; there are very few applications of Javascript that do not fall within the realm of scripting.
ActionScript (the language for Flash animations) and its derivatives are scripting languages, in that they simply issue commands to the Flash player/interpreter. Sure, there are abstractions such as Object-Oriented programming, but all that is simply a means to the end: send commands to the flash player.
Python and Ruby are commonly also used as scripting languages. For example, I once worked for a company that used Ruby to script commands to send to a browser that were along the lines of, "go to this site, click this link..." to do some basic automated testing. I was not a "Software Developer" by any means, at that job. I just wrote scripts that sent commands to the computer to send commands to the browser.
Because of their nature, scripting languages are rarely 'compiled' -- that is, translated into machine code, and read directly by the computer.
Even GUI applications created from Python and Ruby are scripts sent to an API written in C++ or C. It tells the C app what to do.
There is a line of vagueness, of course. Why can't you say that Machine Language/C are scripting languages, because they are scripts that the computer uses to interface with the basic motherboard/graphics cards/chips?
There are some lines we can draw to clarify:
When you can write a scripting language and run it without "compiling", it's more of a direct-script sort of thing. For example, you don't need to do anything with a screenplay in order to tell the actors what to do with it. It's already there, used, as-is. For this reason, we will exclude compiled languages from being called scripting languages, even though they can be used for scripting purposes in some occasions.
Scripting language implies commands sent to a complex software application; that's the whole reason we write scripts in the first place -- so you don't need to know the complexities of how the software works to send commands to it. So, scripting languages tend to be languages that send (relatively) simple commands to complex software applications...in this case, machine language and assembly code don't cut it.
A scripting language is typically interpreted instead of compiled.
See scripting-and-programming,
whats-the-difference-between-a-script-and-an-application
and many similar discussions.
There are many taxonomies for classifying programming languages.
In regard to scripting, a scripting language (python, ruby, php) is a language that runs on an interpretor directly from source code.
Other languages are either compiled languages (run from binary form: c, c++, pascal) either intermediate, compiled to an intermediary form and run inside a virtual machine (java, c#).
A scripting language is a language that focuses on making it easy to chain together or manipulate other programs
Wikipedia says:
"When a language is used to give commands to a software application (such as a shell) it is called a scripting language"
"A scripting language, script language or extension language is a programming language that allows control of one or more software applications."
This is orthogonal from whether it is interpreted or otherwise - Java was originally interpreted, for example, but nobody called it a scripting language. It just happens that, if you're implementing a scripting language, interpreting it is a straightforward approach to take.
Many scripting languages are compiled, either to bytecode or to machine code, often with JIT.
The distinction between script languages and other languages is mostly between interpreted, dynamically typed languages (for example PHP, VBScript, JScript and Javascript), and compiled, statically typed languages (for example C#, VB.NET, Java, C++ and Delphi).
ASP is not a language, but a platform for scripting languages (VBScript / JScript), while ASP.NET is a platform for compiled languages (mainly C# / VB.NET).
What are some situations where languages should be mixed?
I'm not talking about using ASP.NET with C# and HTML or an application written in C accessing a SQL database through SQL queries. I'm talking about things like mixing C++ with Fortran or Ada with Haskell etc. for example.
[EDIT]
First of all: thank you for all your answers.
When I asked this question I had in mind that you always read "every language has its special purpose".
In general, you can get almost everything done in any language by using special libraries. But, if you are interested in learning different languages, why not take the programming language that serves your purpose best instead of a library that solves a problem your language wasn't originally designed for?
For example, in video games we use different languages for different purpose :
Application (Game) code : have to be fast, organized and most of the time cross-platform (at least win, MacOS is to be envisaged), often on constraint-heavy platforms (consoles), so C++ (and sometimes C and asm) is used.
Development Tools : level design tools generates data that the game code will play with. Those kind of tool don't need to run on the target platform (but if you can it's easier to debug) so often they are made with fast-development languages such as C#, Python, etc.
Script system : some parts of the games will have to be tweaked by the designers, using variables or scripts. It's really easier and cheap to embed a scripting language instead of writing one so Lua or other similar scripting languages are often used.
Web application : sometimes a game will require to provide some data online, most often in a database accessed with SQL. The web application then is written in a language that might be C#, Ruby(R.O.R.), Python, PHP or anything else that is good for the job. As it's about the web, you then have to use HTML/Javascript too.
etc...
In my game I use HTML/Javascript for GUI too.
[EDIT]
To answer your edit : the language you know the best is not always the most efficient tool for the work. That's why for example I use C++ for my home-made game because I know it best (I could use a lot of other languages as the targets are Win/Mac/Linux, not consoles) but I use Python for everything related to build process, file manipulation etc. I don't know Python in depth but it's fare easier to do quick file manipulation with it than with C++. I wouldn't use C++ for web application for obvious reasons.
In the end, you use what is efficient for the job. That's what you learn by working in real world constraints, with money, time and quality in mind.
Well, the most obvious (and the most common) situation would be when you use some high level language to make most of your program, reaping the benefits of fast development and robustness, while using some lower level language like C or even assembly to gain speed where it is important.
Also, many times it is necessary to interface with other software written in some other language. A good example here are APIs exposed by the operating system - they're usually written with C in mind (though I remember some old MacOS versions using Pascal). If you don't have a native binding for your language-compiler infrastructure, you have to write some interface code to "glue" your program with "the other side".
There are also some domain-specific languages that are tuned specifically to efficiently express some type of computation. You usually don't write your entire program in them, just some parts where it is the appropriate tool. Prolog is a good example.
Last but not least, you sometimes have heaps of old and tested code written in another language at hand, which you could benefit from using. Instead of reinventing the wheel in a new and better language, you may simply want to interface it to your new program. This is probably the most usual (if not the only) case when languages geared for similar uses are mixed together (was that C++ and Fortran you mentioned?).
Generally, different languages have different strengths and weaknesses. You should try to use the appropriate tool for the job at hand. If the benefits from expressing some parts of the program in a different language are greater than the problems this introduces, then it's best to go for it.
I know in your question you sort of ruled this out, but different languages are used for different domains.
Right now I am working on a data visualizer, the data is in a database so of course there is some SQL, but that hardly counts because it's small and required frequently. The data is turned into a series of graphs, I'm using R, which is like MATLAB but open source. It is a unique statistical language with some advanced plotting features.
A data visualizer isn't just a graph generator, so there needs to be a way to browse and navigate this pile of image files. We opted to use html with embedded javascript to build an offline "application" that can be easily distributed. It's offline in the sense that it is self contained, that html is carefully generated and the js inside it is carefully crafted to allow the user to browse thousands of images sorting or filtering by a number of criteria.
How do you carefully craft javascript and html based on a database structure that changes as the rest of my team makes progress? They are made by a perl program (single pass script really) that reads into the db for some structure and key information, and then outputs over 300 kilobytes of html/js. It's not entirely trivial html either, imagemaps that are carefully aligned with the R plots and some onclick() javascript allow the user to actually interact with a plain image plot so this whole thing feels like a real data browser/visualizer application.
That's four 'languages', five if you count SQL, just to make a single end product.
I dont think doing this in a single language would be a good choice, because we are exploiting the capabilities of a real web browser to give us a free GUI and frontend.
An excellent current example would be to write methods for creating XML documents in VB.NET, which has an "XML Literals" feature, which C# lacks. Since they're both .NET languages, there's no reason not to call one from the other:
Public Function GetEmployeeXml (ByVal salesTerritoryKey As Integer) As XElement
Using context As New AdventureWorksDW2008Entities
Dim x = <x>
<%= From s In context.DimSalesTerritory _
Where s.SalesTerritoryKey = salesTerritoryKey _
Select _
<SalesTerritory
region=<%= s.SalesTerritoryRegion %>
country=<%= s.SalesTerritoryCountry %>>
<%= From e in s.DimEmployee _
Select _
<Employee firstName=<%= e.FirstName %> lastName=<%= e.LastName %>>
<%= From sale in e.FactResellerSales _
Select _
<Sale
orderNumber=<%= sale.SalesOrderNumber %>
price=<%= sale.ExtendedAmount %>/> %>
</Employee> %>
</SalesTerritory> %>
</x>
Return x
End Using
End Function
The biggest reason you would mix langauges is because one language has advantages in certain areas, where another has advantages in another. So, you try to harness the capabilities of both by throwing them together. A common example is using C and ASM, because C is more abstract and makes it easy to do the more complex stuff in a program, however, you would want to use ASM to do base level stuff with hardware and the processor. So, they are often mixed, given the nature of C and its use in embedded systems.
Depending on what paradigm a language falls under, it has its pros and cons. For example, one might want to use the GUI capabilities of C# but use the efficient backtracking or Artificial Intelligence capabilities of Prolog (a language in the logical paradigm) to compute some details before displaying them.
A simplified example
Imagine trying to write a program that allows a user to play Chess against a computer.
(Potentially) I would:
1.) Create the visuals with Windows GUI Libraries.
2.) Calculate the possible moves using Prolog, and choose the best/most viable move.
3.) Retrieve the results from step 2 from Prolog in my C# code, and render the results.
4.) Allow for the gameplay and rapid development of visuals and UI in C# and rely on the Prolog for the calculations/backtracking
This most often happens when you already have code written in two or more different languages and you notice that it makes sense to combine the programs. Rewriting is expensive and takes time.
In the financial world you often have to keep programs alive (or replaced) 50 years. With technology replacement every 10 years, new contracts (mortgages, life insurance) are created in the newest language/environment. The four older ones just handle the monthly payments and changes to existing contracts. To know how the company is doing, you need to integrate data from all five systems.
I suppose keeping the old ones alive is cheaper than migrating each time to the newest technology. From a risk avoiding point of view it makes sense.
Web applications should probably not be mixed language for the developer. Smalltalk does just fine, with Gemstone for persistence and Seaside as web application framework. The multiple languages (javascript) can be hidden in the framework.
When 2 heads are better than one.
I've commonly seen games where flash was embedded with C# - with the AI and other heavy code running off a C++ DLL.
When forced to
Writing new code to augment a new system supported by an old framework
For instance in games (which are pretty hardcore applications) you usually have a very tight c++ engine that does all the heavy lifting and a scripting language (such as Lua) that's accessible and suited for making that collection of special cases that we call 'game' happen.
People here have most of the reason. I'll just have this one:
There is also the case of graceful degradation. For instance I am working on a legacy intranet, and we're changing little by little from a language to another so at this point we have different languages in the same system.
Device Programming - typically you would want to program the UI with the OS's native UI language (Java for Android for example) but would need to program the device drivers with something that gets more into the low level (like C).
From discussions I've had about language design, it seems like a lot of people make the argument that there is not and will never be "one true language". The alternative, according to these people, is to be familiar with several languages and to pick the right tool for the job. This makes perfect sense at the level of a whole project or a large subproject that only has to interact with the rest of the project through a very narrow, well-defined interface.
On the other hand, using lots of different languages seems like a very awkward thing to do when trying to solve lots of small subproblems elegantly. In other words, IMHO, general purpose languages that are decent at everything still matter. As a trivial example, let's say you need to do the following:
Read a bunch of data in some arbitrary format from a file. Check it for errors, etc. (Best done in something like Perl).
Load this data into matrices, do a bunch of hardcore matrix ops on it (Best done in something like Matlab).
Run a custom, computationally intensive routine on it that must be fast and space-efficient (Best done in C or C++).
This is a fairly simple project, other than writing the computationally intensive custom matrix processing routine, yet the only good answer about what language to use seems to be a general-purpose one that's decent at everything.
What am I missing here? How does one use multiple languages effectively to take advantage of each of their strengths?
I have worked on many projects that contain a fairly diverse mix of languages. Unless it's a .NET project, you usually use these different languages at different tiers or in different processes. Maybe your webapp is in PHP and your application server in java. So you do not really "mix and match" at method level.
In .NET and for some of the java vm languages the rules change a bit, since you can mix much more freely. But the features of these languages are mostly defined by the class libraries - which are common. So the motivation for switching languages in .net is usually driven by other factors, such as which language the developers know. F# actually provides quite a few language features that are specific for that language, so it seems to be a little bit of an exception within .NET. Some of the java VM languages also add methods to the standard java libraries, adding features not available in java.
You do actually get quite used to working with multiple languages as long as all of them have good IDE support. Without that I really think I would be lost.
Embedded languages like Lua make this pretty easy. Lua is a great dynamic language along the lines of Ruby or Python and allows you to quickly develop high quality code. It also has tight integration with C, which means you can take advantage of C/C++ libraries and optimize performance critical sections by writing them in C or C++.
In scientific computing it's also not uncommon to have a script that for example will do some data processing in Matlab, use Perl to reformat the output and then pass that into another app written in Matlab, C or whatever. This tends to be more common when integrating apps that have been written by different people than when developing something from scratch, though.
You're saying that the choices are either a) write everything in one language and lose efficiency, or b) cobble together a bunch of different executables in order to use the best language for each section of the job.
Most working programmers pick option c: do the best they can with the languages their IT departments support in production. Generally programmers have some limited language choice within a particular framework such as the JVM or CLR - so it's neither the toolbox utopia nor the monolanguage ghetto.
Even LAMP and Rails support (demand, really) different languages at different levels - HTML, Javascript, Ruby, C in the case of Rails. If you're writing software services (which is where most of the interesting work is happening these days) then you're hardly ever writing in just one language. But your choices aren't infinite, either.
Consider the most difficult part of your task. Is parsing the file the most complex part? A good language for reading files, and then manipulating them might be the most pertinent choice.
Are the math transformations the most difficult part? You might want to suck it up and use matlab, feeding it in with scripts or other home grown tools.
Is performance the most important part? How much calculation will be done versus the transformations and parsing? If 90% of the work done will be on calculation, and the other parts are only transitory, then consider a C/C++ solution.
There's nothing inherently wrong with playing to multiple language strengths in a simple solution, especially if you don't mind gluing it together with scripts. However, the more languages and modules that you integrate increase your dependencies, and the more dependencies you have, the more and more difficult it becomes to distribute and maintain your app. That's where the real trade-off lies: the fewer technologies, libraries, and bits of software you need to run the app, the simpler it becomes. The simpler the app, the more maintainable, distributable, debuggable, and usable it becomes.
If you don't mind the extra overhead of integrating the multiple pieces of technology, or that overhead will save you an equal or greater amount of development time, go right ahead.
I think it's fine. If you set it up as an n-tier setup where each layer accesses what is below it the same way (SOAP/XML/COM, etc), I have found it to be invisible in the end.
For your example, I would choose one language for the front end that is easy, quick, flexible, and can make many, many, many, many, many types of calls to different interfaces such as com, corba, xml, .net, custom dll libraries, ftp, and custom event gateways. This ensures that you can call, or interface with any other lanugage easily.
My tool of choice for this is Coldfusion from working with ASP/PHP/Java and a bit of Ruby. It seems to have it all in one place and is very simple. For your specific case, COldfusion will allow you to compile your own library tag that you can call in your web code. Inside the tag you can put all the c code you like and make it do whatever you want.
There are free and open source versions of Coldfusion available and it is fantastic for making Java and .NET calls naively from one code base. Check it out, I really feel it might be the best fit as it's made for these kinds of corporate solutions.
I know PHP can be pretty good for this too depending on your situation, and I'm sure Asp.net isn't too far behind.
My work in multiple languages has involved just a simple mix of C# and C++/CLI (that's like C++ with .NET if you haven't heard of it). I do think that it's worth saying that the way .NET and Visual Studio allow you to combine many different languages together means that you won't have as high of an overhead as you would normally expect when combining two languages - Visual Studio keeps it all together. Given that there are many .NET clones of various languages - F#, IronPython, IronRuby, JScript.NET for example - Visual Studio is quite a good way in which you can combine various languages.
Down to the technical level which sounds like what you're interested in, the way it's done in Visual Studio is you must have one project for each separate language, and each project gets compiled into an assembly - basically meaning it becomes a library for the other projects to use. You'll have one main project which drives everything and calls the other libraries. The requirement of each different language having to be a different project means you wouldn't change languages at the method level, but if you are doing a large enough application that you can logically separate it into multiple projects, it can actually be quite useful. In particular, the separation and abstraction can make things easier overall.