I would like to understand what is the best way to mitigate risk of vendor lock-in for cloud-based systems.
For example, I'd like to deploy a multitude of different systems to, say, Amazon EC2 or Windows Azure, but I'd like to minimize the cost of migrating those systems to an alternative cloud vendor if/when necessary.
At the very least, it seems like the more I rely on vendor-specific solutions (like Amazon Queue Service), the more I'm inherently locked in (at least I think so), but I'd like to understand this risk better and any beyond it.
Are there architectural strategies I can use to mitigate this (e.g., rely on map reduce, since my scripts will be portable to another map reduce cloud env)? Are there O/S or stacks that are better than others (Linux, LAMP?). Is using JClouds helpful?
Ideally, I'd like to design virtual systems that can be deployed on EC2, for example, but then easily migrated to Azure or App Engine (or vice versa).
I generally write in Java, but am considering selective use of Scala and Python (or Jython) and am generally still trying to stay JVM-based. I tend to do a lot of parallel processing, and rely on both SQL and non-SQL (but not necessary NoSQL) storage and data manipulation technologies.
Thanks in advance. Hope I'm not being too unrealistic here.
In my opinion, the only architectural pattern to the problem you describe is: abstraction
Make sure to stick to using resources that are offered across various vendors,like storage, queue, etc. Create abstraction layers for each of them.
Hope this helps. I don't think its a super simple task to do, given the variability of the services across cloud providers
I agree with IgoreK - if you're doing this in code, it'll take a lot of abstraction, that's about it.
Another option is to take an IaaS cloud approach - design your application based on Virtual machine roles only. Most Cloud providers offer some form of Virtual machine role - Amazon, Azure, Rackspace etc. Migration then means far less code changes, but a bit more admin on your side.
Microsoft's Customer Advisory Team has an excellent sample on how to do that (I think I downloaded the project from here). There's a whole lot of code in it, and some really good abstractions to make things "free". Obviously, as with any abstraction, you also introduce a new layer of complexity so make sure you really all of it before applying it.
In most cases, less is more. And even though a lock-in is not something you want, it's probably not that hard to "fix" if the need arises. But ask yourself if it's important for that need to be satisfied now, or should you finish the project, and refactor later.
Honestly, your question is based on a bit of a false premise. You're looking to avoid lock-in rather than trying to take full advantage of the platform you've chosen to use.
The better way of approaching the issue is not to try to have your infrastructure be hot-swappable (e.g., avoid vendor lock-in), but to actually make a decision about the IaaS provider you want to use and leverage it as best as you possibly can.
Related
Anyone knows an open source project that is on microservices architecture? I need a more real app that has addressed cross-cutting concerns,etc not just an educational sample.
Please introduce if you know any. Especially if it's on Node.js or C#.net stack.
Thanks
As far as I can tell, there are little to no open source projects out there using this pattern!
There are however many great framworks/toolchains to help you implement it:
If you like GO then you're gonna like Go-Kit
If you like C# you might have a look at servicestack.net
as for other languages/toolchains I'm not very well informed but there are many frameworks out there that can help you building microservices in almost every major language.
That said: The main reason for this lack of open source micro service systems is probably that most open source project keep a very narrow track of use cases that they cover to stay generic and reusable. This stands in contrast to what commercial backend systems have to provide - which is myriad of usually very specific business services which will probably never be public since they contain the companies competitive advantages and business critical knowledge! Most large exponents of the micro service pattern (like Netflix, Spotify and SoundCloud) have however open sourced the tools and frameworks they build/used to get the services orchestrated, coordinated, synchronized, health checked, list, balanced, scaled etc...
To give you some general pointer towards micro services in general Martin Fowler has some great resources. Also a good reference are the talks of Peter Bourgon on Micro services.
I will soon be developing an application to log and priorities worker shifts. It only needs to be small, and simple, but I am wondering what framework to use. All that needs to happen is boxes with names are in 3 lists, and the user can switch these around at will. It must log the times, which I will need access to in real time.
Im new to application development of this sort, and would like any suggestions. I have time to learn new technologies / languages.
Portability / device independence would also be useful. Should I consider a Web Application in Javascript? or something more like Python.
Thanks.
Even if your application is going to be simple that does not mean that whole system will be simple too. I can imagine that your app will serve only as a front end to something much bigger. If that is a case and you really have freedom to choose what language you will develop your app with consider choosing something that you will feel comfortable to work with. But before you will make your decision I would go to whoever gave you that task and try to get as much information about it as you can because expected features can help in choosing technology.
First of all, it seems that it is up to you to decide if it should be web or desktop app. In my opinion it is completely wrong situation. You should get clear requirements on what kind of application customer expects and in what environment it should work. And I would not move a finger towards a code before somebody gave me that information. You write that portability and device independence would be useful but is it actual requirement or you just think it would be nice feature to deliver?
EDITED (to answer a question in comment)
Probably there is as much possible solutions as people that would answer you so in the end you will have to make your own choices.
One way of doing it (considering that you want to learn something new and have a challenge :) would be implement WCF service that would act as a data provider from your database (so every GetUsers(), GetVacationDays() methods would be in it) and it would take care of some business logic (for example CalculateMaxValidWorkingTime() or whatever). That service would be a real power horse of your system. Since you don't have clear requirement whether it should be desktop or web app you could satisfy both possibilities by creating thin clients that would communicate with your service and they would be just a pretty front ends. And if you keep in mind that you can consume webservices practically with everything from C++ to .NET (C#/VB) to Javascript to Python to PHP after you done with service you would not be constrained with one particular technology/language.
Regarding databases I won't advice because personally I hate dealing with them and it always was somebody else's task to provide me with pretty API :)
Okay, so we have to store our clients` private medical records online and also the web site will have a lot of requests, so we have to use some scaling solutions.
We can have our own share of a datacenter and run something like Zend Server Cluster Manager on it, but services like Amazon EC2 look a lot easier to manage, and they are incredibly cheaper too. We just don't know if they are secure enough!
Are they?
Any better solutions?
More info: I know that there is a reference server and it's highly secured and without it, even the decrypted data on the cloud server would be useless. It would be a bunch of meaningless numbers that aren't even linked to each other.
Making the question more clear: Are there any secure storage and process service providers that guarantee there won't be leaks from their side?
First off, you should contact AWS and explain what you're trying to build and the kind of data you deal with. As far as I remember, they have regulations in place to accommodate most if not all the privacy concerns.
E.g., in Germany such thing is a called a "Auftragsdatenvereinbarung". I have no idea how this relates and translates to other countries. AWS offers this.
But no matter if you go with AWS or another cloud computing service, the issue stays the same. And therefor, whatever is possible is probably best answered by a lawyer and based on the hopefully well educated (and expensive) recommendation, I'd go cloud shopping, or maybe not. If you're in the EU, there are a ton of regulations especially in regards to medical records -- some countries add more to it.
From what I remember it's basically required to have end to end encryption when you deal with these things.
Last but not least security also depends on the setup and the application, etc..
For complete and full security, I'd recommend a system that is not connected to the Internet. All others can fail.
You should never outsource highly sensitive data. Your company and only your company should have access to it - in both software and hardware terms. Even if your hoster is generally trusted someone there might just steal hardware.
Depending on the size of your company you should have your custom servers - preferable even unaccessible for the technicans in your datacenter (supposing you don't own the datacenter ;).
So the more important the data is, the less foreign people should have access to it in any means. In the best case you can name all people that have access to them in any way.
(Update: This might not apply to anonymous data, but as you're speaking of customers I don't think that applies here?)
(On a third thought: There're are probably laws to take into consideration of how you have to handle that kind of information ;)
In my organisation, we have some very inefficient processes around managing requirements, tracking what was actually delivered on what versions, etc, do subsequent releases break previous functionality, etc - its currently all managed manually. The requirements are spread over several documents and issue trackers, and the implementation details is in code in subversion, Jira, TestLink. I'm trying to put together a system that consolidates the requirements info, so that it is sourced from a single, authoritative source, is accessible via standard interfaces - web services, browsers, etc, and can be automatically validated against. The actual domain knowledge is not that complicated but is highly proprietary and non-standard (i.e., not just customers with addresses, emails, etc), and is relational: customers have certain functionalities, features switched on/off, specific datasources hooked up - all on specific versions. So modelling this should be straightforward.
Can anyone advise the best approach for this - I a certain that I can develop a system from scratch that matches exactly the requirements, in say ruby on rails, grails, or some RAD framework. But I'm having difficulty getting management buy-in, they would feel safer with an off the shelf solution.
Can anyone recommend such a system? Or am I better off building it from scratch, as I feel I am? I'm afraid a bought system would take just as long to deploy, and would not meet our requirements.
Thanks for any advice.
I believe that you are describing two different problems. The first is getting everyone to standardize and the second is selecting a good tool for requirements management. I wouldn't worry so much about the tool as I would the process and the people. Having the best tool in the world won't help if your various project managers don't want to share.
So, my suggestion is to start simple. Grab Redmine or Trac and take on the challenge of getting everyone to standardize. Once you have everyone in the right mindset then you can improve the tools you use for storage.
{disclaimer - mentioning my employer's product}
The brief experiments I made with a commercial tool RequisitePro seemed pretty good me. Allowed one to annotate existing Word docs and create a real-time linked database of the identified requisistes then perform lots of analysis and tracking of them.
Sometimes when I see a commercial product I think "Oh, well nice glossy bits but the fundamentals I could knock up in Perl in a weekend." That's not the case with this stuff. I would certainly look at commercial products in this space and exeperiment with a couple (ReqPro has a free trial, I guess the competition will too) before spending time on my own development.
Thanks a mill for the reply. I will take a look at RequisitePro, at least I'll be following the "Nobody ever got fired for buying IBM" strategy ;) youre right, and I kinda knew it, in these situations, buy is better. It is tempting when I can visualise throwing it together quickly, but theres other tradeoffs and risks with that approach.
Thanks,
Justin
While Requisite Pro enforces a standard and that can certainly help you in your task, I'd certainly second Mark on trying to standardize the input by agreement with personnel and using a more flexible tool like Trac, Redmine (which both have incredibly fast deploy and setup times, especially if you host them from a VM) or even a custom one if you can get the management to endorse your project.
Looking for guidance on how UI design differs for multitenant applications. Pointers to whitepapers and blog entries greatly appreciated.
I can't see why the (graphical) design should be any different. end user shouldn't be able to determine who is hosting the application.
The software design of course has to take care to separate the different instances/clients and to ensure various demands of clients don't affect each other in a negative way.
Also maintenance should be considered. If your app is only used by one client, it is often possible to agree on some downtime for updates, reconfiguration and the like. With many clients this will become different. So you should probably plan for your application to reread configuration without restart. You also should plan for a way to upgrade your application without downtime.
Depends on what type of service you're offering really, doesn't it.
If you're allowing your app to be skinned by various clients (say, like Outlook Webmail) then you need to consider the situation appropriately.
But really, SAAS means nothing here. You just design your ... design, as it should be, and as it needs to be. Simple.