How to find stats for general trends of programming language popularity using google trends - programming-languages

I like to keep an eye on trending browsers/OSs/languages etc... I find google trends is a very useful resource sometimes but other times I can not get the information I want.
Example of very clear increase of Ubuntu (with 6 monthly peaks near release dates) compared with other major linux distros ecline over the years...
http://www.google.com/trends?q=ubuntu%2C+debian%2C+redhat%2C+mandrake&ctab=0&geo=all&date=all&sort=0
Example of results that are skewed because of non-programming related events. See "flash floods" and "earthquake in Java" in the news results
http://www.google.com/trends?q=flash%2C+java%2C+javascript&ctab=0&geo=all&date=all&sort=0
Is there a way to filter the results better to only include Java the programming language, and make sure that it catches all variations of a name - for example js instead of javascript, or an alternative tool that can produce similar graphical trend data.

It is possible to exclude terms with a minus sign and use | for variants:
flash -flood, java -crash -quake, javascript +js
But if you want accuracy it would be better to use the Language Popularity Index (or Tiobe as Bas suggests).

Such "metrics" have questionable value (but are fun to discuss). You could add a word like "program" to each language: http://www.google.com/trends?q=flash+program%2C+java+program%2C+javascript+program&ctab=0&geo=all&date=all&sort=1

Related

Tools for parsing natural language questions in realtime

photos in washington VS show me photos in washington VS I wanna see all my photos in washington taken day before yesterday
what:photos
entities:washington (dont want to be too assuming)
when: 2013-03-14
I want to parse preset queries into conditions (like above). I want these qualities:
I can extract relevant terms even in presence of fluff ("I wanna see) and lowercase nouns
warm program can accept requests over HTTP or allow me to add some network communication
warm program responds in 50ms and needs atmost 500Mb of memory for reasonable sentences
I am more experienced in Python, less so in Java
Parser data structure is easy to handle
I use NLTK, but its slow. I see StanfordNLP and OpenNLP as viable alternatives but I find the program-start latency to be too high. I dont mind integrating them over servlets if I am left with no alternative.
The Stanford Parser is a solid choice, and pretty well-supported (as research code goes). But it sounds like low latency is an important requirement for you, so I'd also suggest you look at the BUBS Parser (full disclosure - I'm one of the primary researchers working on BUBS).
I haven't compared directly to NLTK, but I think you may find that the Stanford Parser doesn't meet your performance needs. This paper found a total throughput of ~60 words/second (~2-3 sentences/second). Those timings are pretty old, so newer hardware will certainly improve that, but probably still won't come close to 50 ms latency.
As you note, startup time will be an issue with any parser - a high-accuracy model is necessarily quite large. And 500 MB is probably pretty tight too (I usually run BUBS with 1-1.2 GB). But once loaded, BUBS latency is generally in the neighborhood of 10 ms per sentence (for ~20-25-word sentences), and we can push the total throughput up around 2500 words/second before accuracy starts to drop off. I think those numbers might meet your performance needs, and I don't know of any other high-accuracy (F1 >= 88-89) parser that comes close in speed.
Note: the fastest results are with recent pruning models that aren't yet posted to the website, but I can get you a model if you need. Hope that helps, and if you have more questions, feel free to ask.

JavaME internationalization (i18n)

Does anyone have some knowledge with internationalization with JavaME? I'm looking for as much information as possible like examples, experiences and maybe some best practices.
Thanks
A few thoughts. J2ME doesnt support i18n as it the api support is not there (cant use resource bundles). But we can do this to a limited extent. Here is what I found out.
It is difficult if not impossible to support english and say chinese languages (typographic characters) for a given J2ME app. But easier to support english and say spanish (I forgot the correct nomenclature to talk about i18n support but you get the idea).
We can have all strings in one config class, that way you can swap this one out for different languages.
We can have the text/strings downloaded from server on initial launch of app and thus have the ability to swap it out from server.
Because of different screen sizes, it is best to work with custom fonts so that code can be written to calculate the text length while displaying it. This will make multiple language support easier.
Image assets can also be downloaded from server based on different languages. But I dont think we can change the midlet icon, so it should be generic.
With this in mind it is possible to design multiple language support.
omemuhammed's answer is an excellent coverages of localization in the mobile space.
I've only had to support EFIGS (English, French, Italian, German, and Spanish). We stored all strings in XML, had an XML pack for each language. We would then compile these XML packs into proprietary binary data and had the ability to build either all 5 languages into a build, have only one language for the build in cases where the application size was tight, or download the binary from a server.
Other considerations with localization is screen layout. I also recommend custom fonts in order to have better control of the display across many different devices. You will need some auto-wrapping code to be able to adjust to different screen resolutions and aspect ratios and you will need a way to handle strings that run off the screen on some devices. Either paging or or scrolling would be a good solution.
Finally, just know that German will screw up your formatting. Try to allow 20-30% padding in English for menus and other UI elements as the German translations will be much longer than the other languages.
See the actual internationalization spec for JavaME: http://www.jcp.org/en/jsr/detail?id=238
Recent Symbian phones should support this.
One obvious advice is to actually try your application on a localized phone: Get a phone from switzerland (it should support at least 4 languages) and another from hong kong (with 3 different version of "chinese"). It might be worth looking into eastern europe/ex-ussr too.
When the actual characters aren't your usual ascii ones, you do need to use a TextBox or TextField in order to have the localized native control on the screen.
Keep in mind that when you use RTL (righ to left) languages, like arabic, you should invert positions of almost everything on the screen, like a list would look like this on latin languages
List item 1
List item 2
List item 3
but the bullets would be on the other side of the screen on arabic (tried make it here, but I couldĀ“n generate na inverted list :P )
One other thing is that is better to store your strings in a class than a plain text properties, as this may cause some errors interpreting the unicode of some languages depending on the OS and text editor you are using.
What I usually do is have a I18nManager Class that stores the language in the startApp and through this class I get all the strings I need.

Major game components

I am in the process of developping a game, and after two months of work (not full time mind you), I have come to realise that our specs for the game are lacking a lot of details. I am not a professional game developper, this is only a hobby.
What I would like to receive help or advices for is this: What are the major components that you find in games, that have to be developped or already exists as librairies? The objective of this question is for me to be able to specify more game aspects.
Currently, we had specified pretty much only how we would work on the visual, completely forgetting everything about game logic (AI, Entities interactions, Quest logic (how do we decide whether or not a quest is completed)).
So far, I have found those points:
Physics (collision detection, actual forces, etc.)
AI (pathfinding, objectives, etc.)
Model management
Animation management
Scene management
Combat management
Inventory management
Camera (make sure not to render everything that is in the scene)
Heightmaps
Entities communication (Player with NPC, enemy, other players, etc)
Game state
Game state save system
In order to reduce the scope of this queston, I'd like it if you could specifically discuss aspects related to developping an RPG type of game. I will also point out that I am using XNA to develop this game, but I have almost no grasp of all the classes available yet (pretty much only using the Game component with some classes that are related to it such as GameTime, SpriteBatch, GraphicDeviceManager) but not much more.
You have a decent list, but you are missing storage (save load), text (text is important in RPGs : Unicode, font rendering), probably a macro system for text (something that replaces tokens like {player} with the player characters name), and most important of all content generation tools (map editor, chara editor, dialog editor) because RPGs need content (or auto generation tools if you need ). By the way have links to your work?
I do this exact stuff for a living so if you need more pointers perhaps I can help.
I don't know if this is any help, but I have been reading articles from http://www.gamasutra.com/ for many years.
I don't have a perfect set of tools from the beginning, but your list covers most of the usual trouble for RUNNING the game. But have you found out what each one of the items stands for? How much have you made already? "Inventory Management" sounds very heavy, but some games just need a simple "array" of objects. Takes an hour to program + some graphical integration (if you have your GUI Management done already).
How to start planning
When I develop games in my spare time, I usually get an idea because another game lacks this function/option. Then I start up what ever development tool I am currently using and try to see if I can make a prototype showing this idea. It's not always about fancy graphics, but most often it's more about finding out how to solve a certain problem. Green and red boxes will help you most of the way, but otherwise, use Google Images and do a quick search for prototype graphics. But remember that these images are probably copyrighted, so only use them for internal test purposes and to explain to your graphic artists what type of game/graphic you want to make.
Secondly, you'll find that you need to find/build tools to create the "maps/missions/quests" too. Today many develop their own "object script" where they can easily add new content/path to a game.
Many of the ideas we (my friends and I) have been testing started with a certain prototype of the interface, to see if its possible to generate that sort of screen output first. Then we build a quick'n'dirty map/level-editor that can supply us with test maps.
No game logic at this point, still figuring out if the game-engine in general is running.
My first game-algorithm problem
Back when I was in my teens I had a Commodore 64 and I was wondering, how do they sort 10 numbers in order for a Highscore? It took me quite a while to find a "scalable" way of doing this, but I learned a lot about programming too.
The second problem I found
How do I make a tank/cannon fire a bullet in the correct direction when I fly my helicopter around the screen?
I sat down and drew quick sketches of the actual problem, looked at the bullet lines, tried some theories of my own and found something that seemed to be working (by dividing and multiplying positions etc.) later on in school I discovered this to be more or less Pythagoras. LOL!
Years and many game attempts later
I played "Dune" and the later C&C + the new game Warcraft (v1/v2) - I remember it started to annoyed me how the lame AI worked. The path finding algorithms were frustrating for the player, I thought. They moved in direction of target position and then found a wall, but if the way was to complex, the object just stopped. Argh!
So I first sat with large amounts of paper, then I tried to draw certain scenarios where an "object" (tank/ork/soldier) would go from A to B and then suddenly there was a "structure" (building/other object) in the pathway - what then?
I learned about A-star pathfinding (after solving it first on my own in a similar way, then later reading about the reason for this working). A very "cpu heavy" way of finding a path, but I learned a lot from the process of "cracking this nut". These thoughts have helped me a lot developing other game algortimes over time.
So what I am saying is: I think you'll have to think more of:
How is the game to be played?
What does the user experience look like?
Why would the user want to come back to the game?
What requirements are needed? Broadband? 19" monitor with 1280x1024?
An RPG, yes - but will it be multi-user or single?
Do we need a fast network/server setup or do we need to develop a strong AI for the NPCs?
And much more...
I am not sure this is what you asked for, but I hope you can use it somehow?
There are hundreds of components needed to make a game, from time management to audio. You'll probably need to roll your own GUI, as native OS controls are very non-gamey. You will probably also need all kinds of tools to generate your worlds, exporters to convert models and textures into something suitable for your game etc.
I would strongly recommend that you start with one of the many free or cheap game engines that are out there. Loads of them come with the source code, so you can learn how they have been put together as you go.
When you think you are ready, you can start to replace parts of the engine you are using to better suit your needs.
I agree with Robert Gould's post , especially about tools and I'd also add
Scripting
Memory Management
Network - especially replication of game object states and match-making
oh and don't forget Localisation - particularly for text strings
Effects and effect timers (could be magical effects, could just be stuff like being stunned.)
Character professions, skills, spells (if that kind of game).
World creation tools, to make it easy for non-programmer builders.
Think about whether or not you want PvP. If so, you need to really think about how you're going to do your combat system and any limits you want on who can attack whom.
Equipment, "treasure", values of things and how you want to do the economy.
This is an older question, but IMHO now there is a better answer: use Unity (or something akin to it). It gives you 90% of what you need to make a game up front, so you can jump in and focus directly on the part you care about, which is the gameplay. When you run aground because there's something it doesn't do out of the box, you can usually find a resource in the Asset Store for free or cheap that will save you a lot of work.
I would also add that if you're not working on your game full-time, be mindful of the complexity and the time-frame of the task. If you'll try to integrate so many different frameworks into your RPG game, you can easily end up with several years worth of work; maybe it would be more advisable to start small and only develop the "core" of your game first and not bother about physics, for example. You could still add it in the second version.

What's the best interactive Analysis and Plotting Tool for software testing?

My realtime app generates a data log: 100 words of data #10Khz. I need to analyze it and produce some plots of the results. There are intermediate calculations involved - I need to take some differences, averages, etc. Excel would work fine, except for:
the 32000 item limit on graph data series is too small - that's only 3 seconds of data.
the glacial speed at which it processes changes to graphs containing large data series is unbearable.
What are good alternatives to Excel for manipulating and plotting large quantities of data? I'm looking for something interactive, not a library.
For this sort of stuff we typically roll our own, but I know that isn't the solution you want. Can you use a good quality database (eg Oracle) to do the manipulation, then maybe put the summarized data back into Excel for the plotting? I believe Excel will link to databases these days, so you could make it quite automated.
Otherwise there are statistical tools like [SAS][1], but get your cheque book out first.
[1]: http://www.sas.com/technologies/analytics/statistics/stat/index.html SAS
There are also several free tools for analysing and plotting (see below). But I am not sure whether they have components to handle data in real-time.
R (similar to SAS) for statistical computations
octave (similar to Matlab) for mathematical computations
R (for data manipulation) and its ggplot2 module for creating sexy graphs. Incredibly useful.
If you need real-time graphics, then I'd look at building something using matplotlib. It's a Python module, and you can link it to R using rpy2 if required.
In particle and nuclear physics the big tool is ROOT, which I have seen used in a "update every two seconds as the data comes in" mode with a lot of data and a modest amount of intermediate processing.
Mind you, the student who wrote that module was a very slick programmer, and it took a while to shake the bugs out, even so.
ROOT is available for free, and provides all kinds of tools and support.

Best way to represent a wide grid for printing?

We have many WIDE html grids which scroll horizontally within a DIV in our web application.
I would like to find the best strategy for printing these grids on a portrait A4 page.
What I would like to know is what is the best way to present/display grids/data like this.
This question is not HTML specific, I am looking for design strategies and not CSS #page directives.
There's actually a whole book dedicated (amongst other things) to fast methods for the computation of \pi: 'Pi and the AGM', by Jonathan and Peter Borwein (available on Amazon).
I studied the AGM and related algorithms quite a bit: it's quite interesting (though sometimes non-trivial).
Note that to implement most modern algorithms to compute \pi, you will need a multiprecision arithmetic library (GMP is quite a good choice, though it's been a while since I last used it).
The time-complexity of the best algorithms is in O(M(n)log(n)), where M(n) is the time-complexity for the multiplication of two n-bit integers (M(n)=O(n log(n) log(log(n))) using FFT-based algorithms, which are usually needed when computing digits of \pi, and such an algorithm is implemented in GMP).
Note that even though the mathematics behind the algorithms might not be trivial, the algorithms themselves are usually a few lines of pseudo-code, and their implementation is usually very straightforward (if you chose not to write your own multiprecision arithmetic :-) ).
I guess it really depends on what your purpose is.
In a book format: I usually try span two facing pages.
For a conference or poster: Find an extra wide printer and print it out on a large sheet of paper.
Something more informal: Span regular pages and tape them together.
Powerpoint: Don't show the whole chart, they'll not be able to read the details anyways, just show the relevant information.

Resources