I've been experimenting with trying to secure a Elasticsearch cluster with basic auth and TLS.
I've successfully been able to do that using Search-Guard. The problem occurs with the Couchbase XDCR to Elasticsearch.
I'm using a plugin called elasticsearch-transport-couchbase which perfectly fine without TLS and Basic Auth enabled on the Elasticsearch cluster. But when enabling that with Search-Guard I am not able to make that work.
As far as I can tell the issue lies with the elasticsearch-transport-couchbase plugin. This has also been discussed previously in some issues on their Github repo.
It is also the only plugin what I can find that can be used for XDCR from Couchbase.
I'm curious about other peoples experience with this. Is there anyone who have been in the same situation as I and been able to setup a XDCR from Couchbase to Elasticsearch with TLS?
Or perhaps there are some other more suitable tools that I can use that I have missed?
The Couchbase transport plugin doesn't support XDCR TLS yet, it's on the roadmap, but isn't going to happen soon. Search-guard adds SSL to the HTTP/REST endpoint in ES, but the plugin opens its own endpoint (on port 9091 by default) which Search-guard doesn't touch. I'll take a look at whether it's possible to extend search-guard to apply to the transport plugin - the main problem is on the Couchbase XDCR side, which doesn't expect SSL on the target endpoint.
Version 4.0 of the Couchbase Elasticsearch connector supports secure connections to Couchbase Server and/or Elasticsearch.
Reference: https://docs.couchbase.com/elasticsearch-connector/4.0/secure-connections.html
A small update. We went around the issue by setting up a stunnel with xinetd. So all communication with ELS have to go through the stunnel where the TLS will terminate.
We blocked access to port 9200, and restricted 9091 to the Couchbase-cluster host and 9300 to the other ELS nodes only.
Seems to work good.
Related
I need to solve an offline synch problem and PouchDB seems to be a good fit.
My client is a PC and my server needs to run on AWS. I am happy to work in node.js on the client.
From reading the documentation, it seems that the recommended configuration would be to use the WebSQL adapter on the client side and CouchDB (or Cloudant) on the server side since it is most "battle proven" adapter.
Q1) Since I am on AWS, would I be better off using an adapter to DynamoDB even if it is less "battle tested"?
Q2) Why isn't running the couchDB adapter also on the client side not a better option than using the WebSQL adapter? Is this even an option?
I have only one server, Solr server. Is it possible to enable Authentication and Authorization for Solr 5 without installing ZooKeeper?
I know that one possible way is to configure, for example, IP table and give access to the server from a certain host (or hosts). But I am interested in Solr's capabilities without any external servers like ZooKeepers.
You can configure your container to do authentication yourself, but the only bundled support in Solr requires running Solr in SolrCloud mode (meaning that it has to either use an external Zookeeper or the internal, bundled one). From [the reference guide about Authentication and Authorization]:(https://cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins)
To use these plugins, you must create a security.json file and upload it to ZooKeeper. This means that authentication and authorization is supported in SolrCloud mode only.
You could also bind Solr to localhost (as Solr shouldn't be exposed on public ips) and then use nginx or Apache to reverse proxy any requests and perform authentication. Configuration would depend on the chosen httpd and how it configures Basic HTTP Authentication.
You don't have to install any external zookeeper to enable authentication and authorization in your solr server. Internal zookeeper works perfectly fine.
http://lucidworks.com/blog/2015/08/17/securing-solr-basic-auth-permission-rules/
I hope this helps.
I feel like I must be really thick here but I'm struggling with couchbase configuration.
I am looking to replace memcached with couchbase and am wanting to secure things more to my liking, on the server their are a number of applications that are set up to use memcached so it needs to be as drop in as possible without changing the applications configuration.
What I have done is installed couchbase on each of the webservers like I did with memcached and so far with my config everything is working.
The problem I have is port 11211 is open to the world a large and this terrifies me, either I'm thick or I'm not looking in the right place but I want to restrict port 11211 to only be listening on localhost 127.0.0.1.
Now couchbase seems to have reams and reams of documentation but I cannot find how to do this and am starting to feel like you need to be a couchbase expert to make simple changes.
I'm aware that the security of couchbase is to use password protected buckets with SASL auth but for me this isn't an option.
While I'm on the subject and if I can change the listening interface, are their any other ports with couchbase that don't need to be open to the world and can be restricted to local host.
Many many thanks in advance for any help, I'm at my wits end.
Let's back up a bit. Unlike memcached, Couchbase is really meant to be installed as a separate tier in your infrastructure and as a cluster even if you are just using it as a high availability cache. A better architecture would be to have Couchbase installed on its own properly sized nodes (in a cluster using Couchbase buckets) and then install Moxi (Couchbase's memcached proxy) on each web server that will talk to the Couchbase cluster on your app's behalf. This architecture will give you the functionality you need, but then give you the high availability, replication, failover, etc that Couchbase is known for.
In the long run, I would recommend that you transition your code to using Couchbase's client SDKs to access the cluster as that will give you the most benefits, performance, resilience, etc. for your caching needs. Moxi is meant more as an interim step, not a final destination.
I am completely new to elasticsearch but I like it very much. The only thing I can't find and can't get done is to secure elasticsearch for production systems. I read a lot about using nginx as a proxy in front of elasticsearch but I never used nginx and never worked with proxies.
Is this the typical way to secure elasticsearch in production systems?
If so, are there any tutorials or nice reads that could help me to implement this feature. I really would like to use elasticsearch in our production system instead of solr and tomcat.
There's an article about securing Elasticsearch which covers quite a few points to be aware of here: http://www.found.no/foundation/elasticsearch-security/ (Full disclosure: I wrote it and work for Found)
There's also some things here you should know: http://www.found.no/foundation/elasticsearch-in-production/
To summarize the summary:
At the moment, Elasticsearch does not consider security to be its job. Elasticsearch has no concept of a user. Essentially, anyone that can send arbitrary requests to your cluster is a “super user”.
Disable dynamic scripts. They are dangerous.
Understand the sometimes tricky configuration is required to limit access controls to indexes.
Consider the performance implications of multiple tenants, a weakness or a bad query in one can bring down an entire cluster!
Proxying ES traffic through nginx with, say, basic auth enabled is one way of handling this (but use HTTPS to protect the credentials). Even without basic auth in your proxy rules, you might, for instance, restrict access to various endpoints to specific users or from specific IP addresses.
What we do in one of our environments is to use Docker. Docker containers are only accessible to the world AND/OR other Docker containers if you explicitly define them as such. By default, they are blind.
In our docker-compose setup, we have the following containers defined:
nginx - Handles all web requests, serves up static files and proxies API queries to a container named 'middleware'
middleware - A Java server that handles and authenticates all API requests. It interacts with the following three containers, each of which is exposed only to middleware:
redis
mongodb
elasticsearch
The net effect of this arrangement is the access to elasticsearch can only be through the middleware piece, which ensures authentication, roles and permissions are correctly handled before any queries are sent through.
A full docker environment is more work to setup than a simple nginx proxy, but the end result is something that is more flexible, scalable and secure.
Here's a very important addition to the info presented in answers above. I would have added it as a comment, but don't yet have the reputation to do so.
While this thread is old(ish), people like me still end up here via Google.
Main point: this link is referenced in Alex Brasetvik's post:
https://www.elastic.co/blog/found-elasticsearch-security
He has since updated it with this passage:
Update April 7, 2015: Elastic has released Shield, a product which provides comprehensive security for Elasticsearch, including encrypted communications, role-based access control, AD/LDAP integration and Auditing. The following article was authored before Shield was available.
You can find a wealth of information about Shield here: here
A very key point to note is this requires version 1.5 or newer.
Ya I also have the same question but I found one plugin which is provide by elasticsearch team i.e shield it is limited version for production you need to buy a license and please find attached link for your perusal.
https://www.elastic.co/guide/en/shield/current/index.html
I want to use Nodejitsu to host my Node.js app. The app also uses Neo4j. Does Nodejitsu provide any kind of support for that? Or do I need to host Neo4j separately (and pay for all network traffic)?
Nodejitsu doesn't support native Neo4j integration. If you can host a Neo4j service or find one you can use it, and for node.js applications you can use the neo4j api with https://github.com/thingdom/node-neo4j
I will try to find the best answer for you, if I can find anything about Neo4j hosting i will update this question.