Related
I am planning to build a web dashboard where I can analyze the financial records from a company through graphics, tables, ...
I already have the software, so the dashboard will only read the data, and not manipulate it.
It will be something like this, but simpler. Containing reports, graphics, options to select dates, intervals, etc.
But I am thinking, is it viable to use Clojure? And jQuery, CSS, HTML also.
Currently I work with the Luminus Web Framework for Clojure, but I am wondering if it is worth to do this in Clojure or if there are other languages that are better to do it.
Of course I am familiar with the language already, so it is a pro. But I am also open to suggestions.
It is not that hard at all! In fact, there exist great libraries which solve all the challenges involved in building a dashboard - scheduling, caching, transferring data to the client, visualization(and auto reloading).
We are working on a framework for building realtime Clojure dashboard. Have a look at https://github.com/multunus/dashboard-clj. We have used the following libraries:
Immutant's scheduler for scheduling
Core.async to simplify data flow on the backend
Sente for websocket communication
re-frame for client side state and view management
Stuart Sierra's component library for managing stateful components
In order to create beautiful visualizations you may take a look at d3 or highcharts. CLJSJS and Reagent cookbook will gives a good overview of how to use these js libraries(and many many more).
Clojure is an absolutely fantastic tool for building a web dashboard. The other answers here do a pretty good job of laying out the landscape as far as basic web technologies. On this side of things, I'll simply add I'm a big Reagent / Re-frame fan, and would go that route for React wrapper over Om.
As far as data visualizations, you may be interested in checking out Vega-Lite & Vega, which you can use from Clojure or ClojureScript (Reagent) by using a simple but flexible dataviz library I wrote called Oz:
https://github.com/metasoarous/oz
Vega-Lite & Vega are designed based on the ideas of the Grammar of Graphics, which inspired R's popular ggplot2 library. The core idea is that data visualizations should be built according to declarative descriptions of how properties of the data map to aesthetics of the visualization. Vega-Lite & Vega however take things one step further in providing a grammar of interaction, which allows for the construction of interactive data visualizations and sophisticated explorer views. Moreover, it ups the ante on the declarative nature of the GG in that Vega-Lite and Vega specifications are described as pure data (JSON), making it very in line with the data-driven philosophy of the Clojure world, and paving the way for seamless interoperability with other languages and sharing features.
Vega-Lite is more or less the higher lever, day-to-day data science tool, focusing on providing high leverage and automation based on very spartan specifications. It compiles to Vega, which is a somewhat lower level and more powerful, but less automated version of Vega-Lite. Usually starting out with Vega-Lite, and switching to Vega only as needed is sufficient.
For more on Vega & Vega-Lite see: https://vega.github.io.
I don't see any reasons why it wouldn't be viable to build a web dashboard in Clojurescript.
I suggest that you look into a library call reagent, which provides a minimalistic interface between react and clojurescript, so theoretically everything you can do with react should be possible in clojurescript/reagent (with the added benefit that it will be faster than React). You probably might be interested in reframe which is a framework for building single page applications.
React has been proven as a robust tool to build powerful UI.
You can do everything you can do in JavaScript using ClojureScript (just as you can do everything you do in Java using Clojure). So as others have commented, I would definitely recommend ClojureScript, especially since you know Clojure already. You may find out that you do not need jQuery etc.
The common choice to generate html is to use React.js via a wrapper library like:
reagent
Om
Both can generate HTML.
Reagent (and maybe re-frame) are the easiest ones to get started. Especially since there are components libraries like soda-ash, and a hiccup-like syntax.
Om (by the creator of ClojureScript), and maybe untangled are also a good choice, especially if you need to manage complex data. You can get a hiccup-like syntax via sablono.
Dashboards have been built using it (see the circleCI dashboard as a real-life dashboard example). This is the one I use personally.
Hoplon is also an interesting choice, as you mentioned.
Also have a look at cljsjs for pre-packaged js libraries.
As for the CSS, this is an orthogonal concern but yes of course you can use it (or even less and sass, there are Clojure wrappers for it). You can even generate CSS from Clojure code with garden,
You can find an example project using boot (by the same authors as hoplon), sass, reagent called saapas, but there are many more in the wild.
As you see there are many viable options in ClojureScript to build a dashboard. I am myself building one and settled on Om.next, partly because I was using React.js before.
What are some situations where languages should be mixed?
I'm not talking about using ASP.NET with C# and HTML or an application written in C accessing a SQL database through SQL queries. I'm talking about things like mixing C++ with Fortran or Ada with Haskell etc. for example.
[EDIT]
First of all: thank you for all your answers.
When I asked this question I had in mind that you always read "every language has its special purpose".
In general, you can get almost everything done in any language by using special libraries. But, if you are interested in learning different languages, why not take the programming language that serves your purpose best instead of a library that solves a problem your language wasn't originally designed for?
For example, in video games we use different languages for different purpose :
Application (Game) code : have to be fast, organized and most of the time cross-platform (at least win, MacOS is to be envisaged), often on constraint-heavy platforms (consoles), so C++ (and sometimes C and asm) is used.
Development Tools : level design tools generates data that the game code will play with. Those kind of tool don't need to run on the target platform (but if you can it's easier to debug) so often they are made with fast-development languages such as C#, Python, etc.
Script system : some parts of the games will have to be tweaked by the designers, using variables or scripts. It's really easier and cheap to embed a scripting language instead of writing one so Lua or other similar scripting languages are often used.
Web application : sometimes a game will require to provide some data online, most often in a database accessed with SQL. The web application then is written in a language that might be C#, Ruby(R.O.R.), Python, PHP or anything else that is good for the job. As it's about the web, you then have to use HTML/Javascript too.
etc...
In my game I use HTML/Javascript for GUI too.
[EDIT]
To answer your edit : the language you know the best is not always the most efficient tool for the work. That's why for example I use C++ for my home-made game because I know it best (I could use a lot of other languages as the targets are Win/Mac/Linux, not consoles) but I use Python for everything related to build process, file manipulation etc. I don't know Python in depth but it's fare easier to do quick file manipulation with it than with C++. I wouldn't use C++ for web application for obvious reasons.
In the end, you use what is efficient for the job. That's what you learn by working in real world constraints, with money, time and quality in mind.
Well, the most obvious (and the most common) situation would be when you use some high level language to make most of your program, reaping the benefits of fast development and robustness, while using some lower level language like C or even assembly to gain speed where it is important.
Also, many times it is necessary to interface with other software written in some other language. A good example here are APIs exposed by the operating system - they're usually written with C in mind (though I remember some old MacOS versions using Pascal). If you don't have a native binding for your language-compiler infrastructure, you have to write some interface code to "glue" your program with "the other side".
There are also some domain-specific languages that are tuned specifically to efficiently express some type of computation. You usually don't write your entire program in them, just some parts where it is the appropriate tool. Prolog is a good example.
Last but not least, you sometimes have heaps of old and tested code written in another language at hand, which you could benefit from using. Instead of reinventing the wheel in a new and better language, you may simply want to interface it to your new program. This is probably the most usual (if not the only) case when languages geared for similar uses are mixed together (was that C++ and Fortran you mentioned?).
Generally, different languages have different strengths and weaknesses. You should try to use the appropriate tool for the job at hand. If the benefits from expressing some parts of the program in a different language are greater than the problems this introduces, then it's best to go for it.
I know in your question you sort of ruled this out, but different languages are used for different domains.
Right now I am working on a data visualizer, the data is in a database so of course there is some SQL, but that hardly counts because it's small and required frequently. The data is turned into a series of graphs, I'm using R, which is like MATLAB but open source. It is a unique statistical language with some advanced plotting features.
A data visualizer isn't just a graph generator, so there needs to be a way to browse and navigate this pile of image files. We opted to use html with embedded javascript to build an offline "application" that can be easily distributed. It's offline in the sense that it is self contained, that html is carefully generated and the js inside it is carefully crafted to allow the user to browse thousands of images sorting or filtering by a number of criteria.
How do you carefully craft javascript and html based on a database structure that changes as the rest of my team makes progress? They are made by a perl program (single pass script really) that reads into the db for some structure and key information, and then outputs over 300 kilobytes of html/js. It's not entirely trivial html either, imagemaps that are carefully aligned with the R plots and some onclick() javascript allow the user to actually interact with a plain image plot so this whole thing feels like a real data browser/visualizer application.
That's four 'languages', five if you count SQL, just to make a single end product.
I dont think doing this in a single language would be a good choice, because we are exploiting the capabilities of a real web browser to give us a free GUI and frontend.
An excellent current example would be to write methods for creating XML documents in VB.NET, which has an "XML Literals" feature, which C# lacks. Since they're both .NET languages, there's no reason not to call one from the other:
Public Function GetEmployeeXml (ByVal salesTerritoryKey As Integer) As XElement
Using context As New AdventureWorksDW2008Entities
Dim x = <x>
<%= From s In context.DimSalesTerritory _
Where s.SalesTerritoryKey = salesTerritoryKey _
Select _
<SalesTerritory
region=<%= s.SalesTerritoryRegion %>
country=<%= s.SalesTerritoryCountry %>>
<%= From e in s.DimEmployee _
Select _
<Employee firstName=<%= e.FirstName %> lastName=<%= e.LastName %>>
<%= From sale in e.FactResellerSales _
Select _
<Sale
orderNumber=<%= sale.SalesOrderNumber %>
price=<%= sale.ExtendedAmount %>/> %>
</Employee> %>
</SalesTerritory> %>
</x>
Return x
End Using
End Function
The biggest reason you would mix langauges is because one language has advantages in certain areas, where another has advantages in another. So, you try to harness the capabilities of both by throwing them together. A common example is using C and ASM, because C is more abstract and makes it easy to do the more complex stuff in a program, however, you would want to use ASM to do base level stuff with hardware and the processor. So, they are often mixed, given the nature of C and its use in embedded systems.
Depending on what paradigm a language falls under, it has its pros and cons. For example, one might want to use the GUI capabilities of C# but use the efficient backtracking or Artificial Intelligence capabilities of Prolog (a language in the logical paradigm) to compute some details before displaying them.
A simplified example
Imagine trying to write a program that allows a user to play Chess against a computer.
(Potentially) I would:
1.) Create the visuals with Windows GUI Libraries.
2.) Calculate the possible moves using Prolog, and choose the best/most viable move.
3.) Retrieve the results from step 2 from Prolog in my C# code, and render the results.
4.) Allow for the gameplay and rapid development of visuals and UI in C# and rely on the Prolog for the calculations/backtracking
This most often happens when you already have code written in two or more different languages and you notice that it makes sense to combine the programs. Rewriting is expensive and takes time.
In the financial world you often have to keep programs alive (or replaced) 50 years. With technology replacement every 10 years, new contracts (mortgages, life insurance) are created in the newest language/environment. The four older ones just handle the monthly payments and changes to existing contracts. To know how the company is doing, you need to integrate data from all five systems.
I suppose keeping the old ones alive is cheaper than migrating each time to the newest technology. From a risk avoiding point of view it makes sense.
Web applications should probably not be mixed language for the developer. Smalltalk does just fine, with Gemstone for persistence and Seaside as web application framework. The multiple languages (javascript) can be hidden in the framework.
When 2 heads are better than one.
I've commonly seen games where flash was embedded with C# - with the AI and other heavy code running off a C++ DLL.
When forced to
Writing new code to augment a new system supported by an old framework
For instance in games (which are pretty hardcore applications) you usually have a very tight c++ engine that does all the heavy lifting and a scripting language (such as Lua) that's accessible and suited for making that collection of special cases that we call 'game' happen.
People here have most of the reason. I'll just have this one:
There is also the case of graceful degradation. For instance I am working on a legacy intranet, and we're changing little by little from a language to another so at this point we have different languages in the same system.
Device Programming - typically you would want to program the UI with the OS's native UI language (Java for Android for example) but would need to program the device drivers with something that gets more into the low level (like C).
I'm lost here, can someone explain to me what subsonic is?
is it a code generator type tool?
"SubSonic is A Super High-fidelity Batman Utility Belt that works up your Data Access (using Linq in 3.0), throws in some much-needed utility functions, and generally speeds along your dev cycle."
Depending on how you use it, it does involve code generation (t4 templates used to create classes that represent your database objects).
This website, Wikipedia, tells me its an ORM (Object-Relational Mapper)
As it's creator Rob Conery says, it's the Swiss Army Knife of ORMS.
Here are some other questions where you can find out about SubSonic:
What ORM is the best when using stored procedures
Best free ORM to use with .Net 3.5
If you haven't go to SubSonic's site and watch some of the tutorials. You find that it is easy to become in under 30 minutes.
I think there is a wealth of natural language data associated with sites like reddit or digg or news.google.com.
I have done a little bit of research with text mining, but can't find how I could use those tools to parse something like reddit.
What kind of applications can you come up with?
I have found in the past that the best way to mine data on sites like Reddit or Digg is to first use the developer API that they provide. Typically you have a focused interest in either a topic or trend, and the only way to get that data is through an established public interface. You can also parse feeds, and combine them both to uncover 90% of what you would want to know. If you want to do deep research on data not available through an API, then you should be prepared to spend a significant amount of time writing custom wrappers around a tool like cURL. If you have the budget you can also call them and ask if they offer paid research data on users.
I'd start on the RSS, and after that I might use Nutch; what to actually do with the data is more your call.
These are good ideas. I can get the data, but what applications can be built around it?
We are developing a web application which is available in 3 languages.
There are these key-value pairs to translate everything. At this moment we use Excel (key, german, french, english) for this. But this does not work well ... if there is more than 1 person editing this file, you have no chance to automatically merge the different files.
Is there a good (and free) tool which can handle this job?
--- additional information ---
(This is a STRUTS application) But the question is how to manage these kinds of information in general (or at least in an conveinient way, which also supports multiple users editing this single file ("mergeable" filetypes))
Why not use gettext and manage separate .po files? See that blog entry.
If you can store this information in plain text then you will be able to use a version control system like subversion to help you with merging changes. Subversion is free.
The free guide (the "Red Book") to subversion gives a fairly good explanation of how this kind of merging works.
http://svnbook.red-bean.com/en/1.5/svn.basic.vsn-models.html#svn.basic.vsn-models.copy-merge
EDIT: Another thought - if you really want to stay using a spreadsheet - Google Docs supports simultaneous editing of a spreadsheet. You could import your existing spreadsheet and get your multi-user merging wishes for free with very little change to how you work.
Good Question.
There are some "Best Practice" depending on what you actually code in (java, ms-windows c#).
I solved this (but I think there must be a better way) by using a SQL db instead of excel file, and a wrote a plug for VS (VB6,........,..., emacs) that was able to insert new keys into the db without going to round trip with version control. The keys are the developers name of what they think is a best guess for a label. (key => save, sv => "spara", no => "", en => "save").
This db can then be generated as a module, class, obj, txt, to appropriate code(platform)
and can be accessed, depending on the ide, so in c#, bt,label = corelang.save;
Someone else can then do all the language stuff, and then we just update the db and rerun the generation to the platform resources.
After years of seeing localization done, including localization at large companies like Sony. I can only say the "standard" is Excel :)
There are tons of good ideas around, and probably many better ways to do it, but in real-life excel seems to be the best/cost effective solution that doesn't require training or making complex new tools to get the job done.
Found out, that Intellij Idea (at leas in version 7 and 8) has an editor for application resources. But it is not free at all. And it does not scale for bigger resource files with more than 1.000 keys.
Another good choice would be to use Google's spreadsheets ... for those who don't know it - it is like an "online Excell web-application". It can handle concurrent access from multiple users. Yay! But sadly, it comes from Google. This makes it impossible to be used in commercial projects.
So,
still searching...
cheers,
mana