I am looking to create an AI environment where users can submit their own code for the AI and let them compete. The language could be anything, but something easy to learn like JavaScript or Python is preferred.
Basically I see three options with a couple of variants:
Make my own language, e.g. a JavaScript clone with only very basic features like variables, loops, conditionals, arrays, etc. This is a lot of work if I want to properly implement common language features.
1.1 Take an existing language and strip it to its core. Just remove lots of features from, say, Python until there is nothing left but the above (variables, conditionals, etc.). Still a lot of work, especially if I want to keep up to date with upstream (though I just could also just ignore upstream).
Use a language's built-in features to lock it down. I know from PHP that you can disable functions and searching around, similar solutions seem to exist for Python (with lots and lots of caveats). For this I'd need to have a good understanding of all the language's features and not miss anything.
2.1. Make a preprocessor that rejects code with dangerous stuff (preferably whitelist based). Similar to option 1, except that I only have to implement the parser and not implement all features: the preprocessor has to understand the language so that you can have variables named "eval" but not call the function named "eval". Still a lot of work, but more manageable than option 1.
2.2. Run the code in a very locked-down environment. Chroot, no unnecessary permissions... perhaps in a virtual machine or container. Something in that sense. I'd have to research how to achieve this and how to make it give me the results in a secure way, but that seems doable.
Manually read through all code. Doable on a small scale or with moderators, though still tedious and error-prone (I might miss stuff like if (user.id = 0)).
The way I imagine 2.2 to work is like this: run both AIs in a virtual machine (or something) and constrain it to communicate with the host machine only (no other Internet or LAN access). Both AIs run in a separate machine and communicate with each other (well, with the playing field, and thereby they see each other's positions) through an API running on the host.
Option 2.2 seems the most doable, but also relatively hacky... I let someone's code loose in a virtualized or locked down environment, hoping that that'll keep them in while giving them free game to DoS or break out of the environment. Then again, most other options are not much better.
TL;DR: in essence my question is: how do I let people give me 'logic' for an AI (which I think is most easily done using code) and then run that without compromising the functionality of the system? There must be at least 2 AIs working on the same playing field.
This is really just a plugin system, so researching how others implement plugins is a good starting point. In particular, I'd look at web browsers like Chrome and Safari and their plugin systems.
A common theme in modern plugins systems is process isolation. Ideally you should run the plugin in its own process space in a sandbox. In OS X look at XPC, which is designed explicitly for this problem. On Linux (or more portably), I would probably look at NaCl (Native Client). The JVM is also designed to provide sandboxing, and offers a rich selection of languages. (That said, I don't personally consider the JVM a very strong sandbox. It's had a history of security problems.)
In general, my preference on these kinds of projects is a language-agnostic API. I most often use REST APIs (or "REST-like"). This allows the plugin to be highly restricted, while not restricting the language choice. I like simple HTTP for communications whenever possible because it has rich support in numerous languages, so it puts little restriction on the plugin. In fact, given your description, you wouldn't even have to run the plugin on your hardware (and certainly not on the main server). Making the plugins remote clients removes many potential concerns.
But ultimately, I think something like your "2.2" is the right direction.
I need to develop a large set of tools to be run from the server command line (i.e. not client-server architecture). The systems does not have to be high-performance; I just want something that is easy to develop with.
Which technologies are out there I can use to build simple GUI to be run from the command line? I need only menus where I can select a line/check-box/enter free text in a dialog.
Edit: forgot to add, access to Mysql (i.e. drivers available) is essential.
Shell, with dialog, the old stand-by - http://www.linuxjournal.com/article/2807
EDIT- If it's MySQL-related, take a look at PERL-Tk and DBI.
python + ncurses would be a good combo here.
i like using perl's re.pl from the Devel::REPL library for quickie cli interfaces. read on a bit for my rationale before downvoting!
in this type of app it sounds like you will be doing query-type operations. these naturally lend themselves to a "repl" style interraction. re.pl gives you all of the goodies, namely command editing and history. all you need to write are the functions that users will call. the nice thing is that users who know perl will realize they can use any installed module to extend the functionality of your system on their own. i my case, i used re.pl to create a mysqlclient-like tool to access and display data that was being compressed in a way that the standard mysqlclient couldn't deal with.
i cite perl because it's DBI is the best database abstraction and it is what i have used....but the rationale can be extended to other tools. python's repl or any other would provide the same benefit.
You could use Mono for Linux and write your program in C# .NET, then make it work for Linux, since Mono allows so.
As far as graphic command line interfaces go, one of the best frameworks is ncurses. It abstracts away most of the ugliness associated with graphic command line applications.
I have to say, use Python, because I like it.
But text-based interfaces are pretty much not worth it, because they seem like a good idea until you look at the details:
There isn't really a standard keyboard navigation model for text-UIs; they all use their own scheme
How is unicode supported? (Hint: this is nontrivial)
What about different keyboard layouts? What key does someone press if their keyboard doesn't have, say, a "home", "end", or "Escape" ?
ncurses does not provide a widget set, only low-level operations. The answers to the above questions aren't easy.
It really shows that nobody has put much thought into keyboard-and-text-driven terminal-based UIs recently, or these would all have been solved.
Web interfaces have them solved, in fact, you can use a text-mode web browser if you like.
Modern devices like i(phone|pad)s and even cheap mobile phones have a web browser which is good enough.
It is easy to write a web application which uses a very simple style (few images, little Javascrfipt) and have it work without much effort on a variety of devices.
So I would say go with dmckee's comment "go with what you know".
By building your own terminal-based interface, you are going to box yourself into a corner in the long term.
This may not be for this forum, but...
We write new system and people are used to older system where components are called "modules". So they talk about the accounting module and the auditing module, etc...
This feels very old, like cobol/mainframe talk. What would be better term for functional components in a modern-distributed java system? Would you say the accounting component? The accounting service? Not sure. They refer to the function in the system (and all components behind it) that allow you to perform accounting functions.
If it ain't broke, don't fix it.
The fact that you are asking SO for advice on this suggests that you don't have a better nomenclature ready to use. Spend your time doing something more productive than fretting about this.
I think that "module" is a perfectly reasonable way to refer to a set of functionality. It's still widely used in many languages and frameworks. If it sounds "old" it's only because of your own frame of reference.
Besides, the customer is always right. You should be adopting their verbiage instead of trying to force them to use yours. Do what you want internally but stick with "modules" for the customers' sakes.
Python, a thoroughly modern programming language, has "modules." I don't think there is anything archaic about the word.
If users are used to calling things in the old system "modules," then it will make the new system easier to learn if you use the same terminology.
bundle is the new term ;-) ofcourse coined by OSGI. But when you say bundle then lot of assumptions are made on your code. So whether you want to use this or not is left to you.
I work on an application that has a both a GUI (graphical) and API (scripting) interface. Our product has a very large installed base. Many customers have invested a lot of time and effort into writing scripts that use our product.
In all of our designs and implementation, we (understandably) have a very strict requirement to maintain 100% backwards compatibility. A script which ran before must continue to run in exactly the same way, without any modification, when we introduce a new software version.
Unfortunately, this requirement sometimes ties our hands behind our back, as it really restricts our ability to innovate and come up with new and better ways of doing things.
For example, we might come up with a better (and more usable) way of achieving a task which is already possible. It would be desirable to make this better way the default way, but we can't do this as it may have backwards compatibility implications. So we are stuck with leaving the new (better) way as a mode, that the user must "turn on" before it becomes available to them. Unless they read the documentation or online help (which many customers don't do), this new functionality will remain hidden forever.
I know that Windows Vista annoyed a lot of people when it first came out, because of all the software and peripherals which didn't work on it, even when they worked on XP. It received a pretty bad reception because of this. But you can see that Microsoft have also succeeded in making some great innovations in Vista, at the expense of backwards compatibility for a lot of users. They took a risk. Did it pay off? Did they make the right decision? I guess only time will tell.
Do you find yourself balancing the conflicting needs of innovation and backwards compatibility? How do you handle the juggling act?
As far is my programming experience is concerned, if I'm going to fundamentally change something that will prevent past incoming data to be used correctly, I need to create an abstraction layer for the old data where it can be converted for use in the new format.
Basically I set the "improved" way as default and make sure through a converter it can read data of the old format, but save or store data as the new format.
I think the big thing here is test, test, test. Backwards compatibility shouldn't hinder forward progress.
Split development into two branches, one that maintains backwards compatibility and one for a new major release, where you make it clear that backwards compatibility is being broken.
The critical question that you need to ask is wether the customers want/need this "improvement" even if you perceive it as one your customers might not. Once a certain way of doing things has been established changing the workflow is a very "expensive" operation. Depending on the computer savyness of your users it might take some a long time to adjust to the change in the UI.
If you are dealing with clients innovation for innovation's sake is not always a good thing as fun as it might be for you to develop these improvements.
You could alawys look for innovative ways to maintain backwards compatibilty.
Recently I've been doing lots of weekend coding, and have began to really need a bugtracker as things are gaining speed. This is probably the worst case scenario because I basically have to let things cool down over the week,so I simply can't remember the bugs in my head. So far I've been using a text file to jot down bugs,but I'd rather use something a bit better.
The biggest points here are ease of use and very little setup time.Don't want to spend more than an hour learning the basics and trying to install something. Also in my case I'm on a Mac so that would help, but solutions for other platforms are welcomed as they will likely help others.
FogBugz has a student/startup edition that's free indefinitely, for 2 or less users.
Personally, I use Excel. (Wait, come back, I'm not crazy!) For a bigger / team project, I've gotten a ton of mileage out of Bugzilla, but that tends to be kind of overkill for a one-person project.
But, a well-organized spreadsheet, with columns for things like "status", "description", "code module", "resolved date," etc, gets you pretty close to what you'd need for a small project. Sorting a spreadsheet by column isn't anywhere near a search, but its a whole lot better than "find in text file."
Heck, if you use Google docs rather than excel, you can even publish the thing as an RSS feed and get it anywhere.
And, the major advantage is that the setup time and learning curve are both effectively nil.
Addendum: And of course, the instant your "One-Person Bug Tracker" becomes a "Two-Person Bug Tracker" you must switch to something better. Bugzilla, FogBugz, anything. Trust me, I've been there.
Trac or Redmine are both pretty good. I don't know how easy they are to set up on a Mac.
It's worth mentioning that FogBugz also has a free version for up to 2 users, which would suit you. It is hosted so there is no installation and you can use something like Fluid to access it in its own window.
I don't think you need a full blown bugtracker for your scenario.
Try tiddly wiki, store each bug in a tiddler and give them tags like 'open' or 'closed'.
There is no installation required (only one html file), and it's very easy to use.
And platform neutral.
If you're working on a LAMPP stack, then for ease of setup and use I would probably recommend Mantis. It's written in PHP / MySQL and the only installation involved was specifying where the database should be created and what credentials should be used.
Oh, and its FOSS.
I would suggest Omnigroup's Omnifocus - it's an excellent task tracker, and if you just make the mental leap from bug to task, I think it works famously for one man projects as well as being an excellent way to organize your no doubt burgeoning task queue.
Eclipse has a really interesting system--I don't know why so few people seem to know about it.
It's tied in with their to-do list. It gives you the ability to enter bugs with as much or as little info as you like. You can tie it to versioning or an external bug tracker if you like. It's a decent bug tracker in itself.
The real trick is how it works with your source code.
Before you begin work you select a bug from the list. All the time you're coding, it tracks what files you are editing. It can close old tabs for you, and will also highlight areas of the source tree that you have modified a lot.
The nice thing is, you can go back to any bug you've edited an you will get your "Environment" back. Not only all your notes and stuff, but the same tabs will open up and the same sections of code in the navigator will be highlighted.
Also eclipse works with virtually any language, it's not just restricted to Java...
let me put in a good word for ditz - it's a bit bare-bones, but it has the invaluable feature that bugs are checked into your repository. it's also very easy to use once you get used to its way of doing things
You can use fogbugz for free if you're a one man team.
It's super easy to use and quick to learn.
They made it so that bugs are really easy to enter, no mandatory fields.
I'm the author of BugTracker.NET mentioned in another post. If I were looking for a tracker for JUST ONE PERSON with MINIMUM hassle, I'd use FogBugz, because it's hosted. No installation, no need to worry about backups.
But, what are you doing about version control? Don't you have to worry about that too, and backing that up? If so, consider something like Unfuddle or CVSDude where you can get BOTH Subversion and Trac, or Subversion and Fogbugz.
I use Mantis at home and I'm happy with it. It can be a pain in the arse to get it working so you can choose to download a free and ready-made VM installation. Cannot be easier than that,
Maybe a spreadsheet would be the next logical step? I know it sounds really un-sexy, but if you're the only user, you don't have to worry much about others mucking it up, and it adds a few basic features over a text file like sorting. Then if you later need to graduate to something RDBMS-backed, you would likely have a feasible import path. I just know that for me, when working by myself, I don't tend to get around to putting bugs in anything that requires more care and feeding than that (of course when working with others the collaborative needs make a more defined repository a requirement, but that's a different story).
EDIT: After noting the availability of free, hosted access to FogBugz, I'm re-thinking the bar for care and feeding...
RT from BestPractical is great.
I also get a lot of mileage out of just keeping a list of items in a text file with vi, if I can express them all in one line. This is usually for many small todo items on a single component or task.
I've tried bugtracker.net and even though it's a little bit rough on the edges, it's free and was built with ASP.NET:
http://sourceforge.net/project/showfiles.php?group_id=66812
Are you using a source control repository as well? If not, you really should, even though you're only a one-man team.
My personal preference is to use a VMWare Virutal Application (free) that offers no-hassle setup gives you access to both Trac and Subversion. You can find many different virual appliances through searching. Here is one example of getting a Trac/SVN virtual appliance up and running:
http://www.rungeek.com/blog/archives/how-to-setup-svn-and-trac-with-a-virtual-appliance/
Trac is an excellent project management tool that sports a bug tracker, wiki, and integrated source control management. It's adaptable to your needs, and fits me very well personally.
I use bugzilla for this purpose. Plus for me was that it has integration with Eclipse (precisely with Mylyn). FogBuzz has it to but AFAIK it is nonfree.
Plus it sits on my laptop so I can code and add/remove bugs when offline (it was biggest disadvantage of hosted solutions for me)
Installation was not a problem in Ubuntu (and any debian-based distro I suppose).
I dig ELOG in those cases, it's more of a personal blog, but it's easy to handle and install, the data is local on your computer and you can search all entries via fulltext. Always sufficed for me.
If you have a Windows box with IIS and MSSQL (including SQL Server Express), you should look at Bugtracker.net. It is free and open source (you get the source code), and it is extensible.
Even if you are a one man shop, having a free bug tracking system with this much power will allow you to grow over time, because it is fairly easy to add future users into the system.
You can also customize it for the look of your organization, business or product.
Ontime 2008 by Axosoft is free for a single user licence. It's industrial strength and will give you alot more that just bug tracking!
http://www.axosoft.com
Jira which now has free personal licenses.
I am using leo for this purpose. To be more specific, its cleo plugin.
Of course you might need to spend some time to get used to leo, but it will pay off.
A flat text file is just a list, an Excel spreadsheet is a two-dimensional list.
leo lets you keep the data in a tree! And it also has clones.