We have an application running on a loadbalanced environment, let say webserver A and B.
The loadbalancing is on the HTTP level, so the loadbalancer directs each user request to one of both webservers.
The scope of the repositories in the application is managed by the spring.net container,
and the application relies on data that can be cached by the repository (performance reasons).
In this case we can never be sure that the cached data in the repositories on both webservers is the same.
Is there mechanism in spring.net that can manage this kind problem? Or is there another common approach for this kind of thing?
Any ideas?
Thx,
Bert
Spring.NET is an object container. It is not intended to be used as a caching container therefore you should not store your data in the repositories. What you need in this case is a distributed caching solution such as memcached. It has a .NET client library also.
So basically your repositories will first look in the distributed cache if the object exists and if not then hit the real data store in order to get the object and store it into the cache. Spring.NET will handle the lifetime of these repositories and as you are running in a webfarm you will have a repository per server but this won't be an issue because these repositories will use a distributed cache to fetch data so you will get consistent resylts between all the servers in the farm.
Related
I want to create and host 4-5 websites using the same database. The only difference between the sites will be:
branding (colours and header)
data will be filtered per website (through sql query) and
Each site will be on a separate domain (but can be hosted on same server)
My 1st thought was to use API / Rest model and provision five front-ends in their own sub-domain. But as sites can be hosted on same server (I'm assuming one hosting account which enables multiple sub-domains), I think I can simply connect all sites with connection string to same database, avoiding complexities of using REST.
Is this possible and would i run into database conflicts doing this?
If later, I wanted to add a mobile app client, then will I need to build out a rest interface anyway?
Thanks
The right thing to do here depends a lot on your specific use case, expected load, preferred backend/edge technology, future plans, etc.
Site domains and servers -
The main point here is that you can host your domains/subdomains on the same or different servers. You simply need to update the DNS to point to the correct IP (update the subdomain's A record).
Note: If these sites are all public-facing, then I highly recommend using an edge/proxy server and even consider a load balancer, depending on expected number of visitors (Nginx, or Apache Web Server)
Decoupled architecture is almost always preferred -
I would definitely have an API/REST layer to abstract the database from the sites. This ensures that you establish a contract through which any clients can interact with the backend, including your mobile application. You also don't have to duplicate DB-specific code across the various clients. What if you decided to change your schema? Or even your database solution? Then all clients will be broken and your customers would be unhappy. As a guiding principle, think: if I change any one thing in my architecture, how many other things will need to change as a result? In terms of scalability, this architecture will also allow you to easily spin up more instances of whatever it is you need (databases, REST service, etc) should the need arise.
How do I build and deploy a REST API?Re: #2, to set up a simple custom REST service running on Node.js (and express), this is a good tutorial. The example also walks through setting up and integrating with an in-memory MongoDB database.
Database collisions?If you follow the above steps, this should be a moot point. Node.js/express and the databases expose ways to configure connection pools if the defaults do not suffice. Again, this will depend on your needs - how many concurrent users you expect.
There are multiple node services currently deployed and running through pm2 in aws environment.
Difficulty(in terms of maintenance) I see in my current code base is that each of these node services have a a separate configuration file (config\app.json) - Though, most of the properties in these configuration files are common for all the services, each of the property is mentioned in each individual service in code. If there is a change is any of these properties, I will have to modify the change in multiple places.
I would like to centralise the configurations across multiple node services. Is there a way to do that? Expectation is to have a centralised place for maintaining configurations. Any references would help.
I am not sure how your architecture is but if you do not mind creating a small library or microservice, which will just fetch you configurations from a small NoSQL database such as Redis which stores key-value pairs, then it will provide you with configurations at a centralized place.
Now the only configuration remains here is of redis which you can add while building the service by providing it's configuration as an environment variable using some thing like yargs.
Then in every service you'll have to make only one API call to fill up your config json in your case config/app.json
I want to use hazelcast as a local-only inter-JVM shared cache. Or put another way, I want to run a secure/private instance.
Is this possible? If so, how?
If it matters, it will be spring-managed.
The motivation is that confidential data will be stored in hazelcast and I want to protect it from external attacks.
You need to define your own Group configuration credentials which would be needed to connect to your cluster.
<hz:group name="dev" password="password"/>
Best practices:
Always define your own Hazelcast xml/spring configuration instead of using the default one from the jar file
Better to make use of TCP/IP network configuration wherever possible instead of multicast, so your cluster won't collide with others.
Define custom group credentials, as mentioned above.
I am completely new to elasticsearch but I like it very much. The only thing I can't find and can't get done is to secure elasticsearch for production systems. I read a lot about using nginx as a proxy in front of elasticsearch but I never used nginx and never worked with proxies.
Is this the typical way to secure elasticsearch in production systems?
If so, are there any tutorials or nice reads that could help me to implement this feature. I really would like to use elasticsearch in our production system instead of solr and tomcat.
There's an article about securing Elasticsearch which covers quite a few points to be aware of here: http://www.found.no/foundation/elasticsearch-security/ (Full disclosure: I wrote it and work for Found)
There's also some things here you should know: http://www.found.no/foundation/elasticsearch-in-production/
To summarize the summary:
At the moment, Elasticsearch does not consider security to be its job. Elasticsearch has no concept of a user. Essentially, anyone that can send arbitrary requests to your cluster is a “super user”.
Disable dynamic scripts. They are dangerous.
Understand the sometimes tricky configuration is required to limit access controls to indexes.
Consider the performance implications of multiple tenants, a weakness or a bad query in one can bring down an entire cluster!
Proxying ES traffic through nginx with, say, basic auth enabled is one way of handling this (but use HTTPS to protect the credentials). Even without basic auth in your proxy rules, you might, for instance, restrict access to various endpoints to specific users or from specific IP addresses.
What we do in one of our environments is to use Docker. Docker containers are only accessible to the world AND/OR other Docker containers if you explicitly define them as such. By default, they are blind.
In our docker-compose setup, we have the following containers defined:
nginx - Handles all web requests, serves up static files and proxies API queries to a container named 'middleware'
middleware - A Java server that handles and authenticates all API requests. It interacts with the following three containers, each of which is exposed only to middleware:
redis
mongodb
elasticsearch
The net effect of this arrangement is the access to elasticsearch can only be through the middleware piece, which ensures authentication, roles and permissions are correctly handled before any queries are sent through.
A full docker environment is more work to setup than a simple nginx proxy, but the end result is something that is more flexible, scalable and secure.
Here's a very important addition to the info presented in answers above. I would have added it as a comment, but don't yet have the reputation to do so.
While this thread is old(ish), people like me still end up here via Google.
Main point: this link is referenced in Alex Brasetvik's post:
https://www.elastic.co/blog/found-elasticsearch-security
He has since updated it with this passage:
Update April 7, 2015: Elastic has released Shield, a product which provides comprehensive security for Elasticsearch, including encrypted communications, role-based access control, AD/LDAP integration and Auditing. The following article was authored before Shield was available.
You can find a wealth of information about Shield here: here
A very key point to note is this requires version 1.5 or newer.
Ya I also have the same question but I found one plugin which is provide by elasticsearch team i.e shield it is limited version for production you need to buy a license and please find attached link for your perusal.
https://www.elastic.co/guide/en/shield/current/index.html
We want to use Milton WebDav to transfer files in our web application which eventually is going to be deployed on cloud environment (most likely azure) as IaaS.
Now we are aware that WebDAV standard is stateless and hence it should not create any problems with cloud load balancer, but what we are not sure about Milton and have few questions:
1.) Is Milton implemented WebDAV as it is, do all the communication remains stateless? I assume that it passes Authentication token with every request but I am not sure where is the token stored at server? Does it store it in the database or some sort of cache etc.?
2.) Do locking mechanism works fine if a load balance is used and there are 5-6 servers to handle the load? Again where does Milton server store Lock Token?
Sorry for the late comment, the two most important aspects of webdav which affect load balancing are digest authentication tokens (Nonce values) and lock tokens.
As the Resource implementor you get to control both of those. Lock tokens are typically stored in a database (you must implement the methods on LockableResource which will do the persistence) so will be shared across servers, although its not uncommon to use memory based lock tokens, in which case you need to find some way to share that information across servers.
Digest nonces are only a consideration if you've implemented DigestResource. The default NonceProvider uses a simple HashMap so this will not be shared across servers. But the interface is trivial so you can easily implement a database store. If your load-balancing solution uses sticky sessions then that won't be an issue because clients will go to the server which has their nonce.
Note that Tomcat session replication won't help with the above issues, because webdav clients typically dont support cookies, so there is no Servlet session.
I have never used Milton WebDAV before but from the looks of it, it is used to modify and edit files on a server.
However Azure's local storage is not shared. Each instance is a completely seperate server. If you modify a file on 1 server, it will not be replicated to the next.
Azure works by uploading a deployment package. When a new instance needs to come up it uses the deployment package and starts a completely new server.
From a your perspective they don't share anything in common. Because of this you will never know which server you are hitting.
If you have a shared file storage system behind, then it may be a different story. However that scenario looks odd from using Azure. Amazon EC2 with a shared EBS might do it though.