I’m currently working on a docker-compose setup that can be used to deploy a cluster of CouchDB 2 nodes. I’ve finally got the nodes working and the data syncing across nodes, but unless I am mistaken, it appears that CouchDB does not sync user sessions.
My setup has 3 nodes and uses an haproxy setup almost identical to haproxy.cfg. As per my configuration, haproxy routes incoming traffic on port 5984 to port 5984 on all 3 nodes.
Assume an admin username of root and password of password.
I first log in with:
curl -vX POST http://localhost:5984/_session -H 'Content-Type: application/x-www-form-urlencoded' -d 'name=root&password=password'
Note the returned AuthSession is used below as AUTHSESSION.
Then, I issue the following:
curl -X PUT http://localhost:5984/mydb --cookie AuthSession=AUTHSESSION -H "X-CouchDB-WWW-Authenticate: Cookie" -H "Content-Type: application/x-www-form-urlencoded"
This usually fails with “You are not a server admin.” I can continue to issue the same PUT and it will eventually succeed as I assume that haproxy eventually routes the request to the single node with which I am authenticated. As haproxy is using round robin there is a 1 in 3 chance that I will hit the target node.
I would think that CouchDB 2 could handle syncing user sessions across nodes. Am I making a silly assumption here?
(Please see run cluster via docker-compose to replicate my setup)
Update with specific solution for my docker-compose setup
As per #lossleader, you need to set the secret in the [couch_httpd_auth] section so that it is identical across nodes. Moreover, you need to set the same admin username and password in the [admins] section. The detail I missed here is that all nodes must have the exact same password hash in the .ini file. Having the same cleartext password is not enough, as otherwise, each node will generate its own salt and generate a different hash.
See run cluster via docker-compose for my complete setup.
Short answer: yes.
Long answer:
As others have commented, couchdb doesn't know the sessions its made, so it's true there's no mechanism to sync sessions themselves, but there are two non-session things that you need to sync yourself before a session cookie made on one node of a cluster will be valid on any other.
[couch_httpd_auth]
secret = foo
This is the secret value used to sign session cookies. If not present when a session cookie is requested, it is set to a random value. Each node of a cluster will, naturally, generate a different value.
So, before startup, arrange for this value to be set to a large, random value but the same on all nodes of your cluster.
[admins]
foo = -pbkdf2-2cbae77dc3d2dadb43ad477d312931c617e2a726,cd135ad4d6eb4d2f916cba75935c3ce7,10
This section contains the salted password hashes of each admin user. The salt is included in the signature in the session cookies. On password change, the salt is re-randomized, so the effect of including salt is that it invalidates the session cookies from before the password change.
You also need this section to be the same on all nodes. Each node will generate a random salt when hashing the admin password.
Better to generate this section externally as part of your node provisioning automation.
I hope that gets you started. We would like to improve this situation in a future release, it obviously reflects the pre-clustering versions of couchdb.
CouchDB session tokens are just an HMAC hash of the user's password salt, the server secret, and the time. Sessions aren't stored in CouchDB at all, even on a single-node system. So there is nothing to sync.
You can, and many people do, generate sessions entirely programatically, external to CouchDB.
Related
My aim is to log unique queries per session by writing custom QueryHandler implementation as logging all queries causes performance hit in our case.
Consider the case : If a user connected to cassandra cluster with java client and performs "select * from users where id = ?" 100 times.
And another user connected from cqlsh and performed same query 50 times. so i want to log only two queries in this case. For that i need a unique session id per login.
Cassandra provides below interface where all requests lands up but none of its apis provide any sessionId to differentiate between two different session described in above case.
org.apache.cassandra.cql3.QueryHandler
Note: I am able to get remoteaddress/port but i want some id which is created when user logged in and get destroyed when he disconnects.
In queryState.getClientState().getRemoteAddress() the address + port will be unique per tcp connection in the sessions pool. There can be multiple concurrent requests over each connection though, and a session can have multiple connections per host. There is also no guarantee the same tcp connection will be used from one request to another on client side.
However a single session cannot be connected as 2 different users (part of the initialization of connection) so the scenario you described isn't possible from the same Session object perspective. I think just using the address as the key for uniqueness will be all you can do given how the protocol/driver works. It will at least dedup things a little.
Are you actually processing your logging inline or are you pushing it off async? If using logback it should be using async appender but if your posting events synchronously to another server, might be better just to throw all the events on a queue and let it do the deduping in another thread so you don't hurt latency.
I have just started using beanstalkd and pheanstalk and I am curious whether the following situation is a security issue (and if not, why not?):
When designing a queue that will contain jobs for an eventual worker script to pick up and preform SQL database queries, I asked a friend what I could do to prevent an online user from going into port 11300 of my server, and inserting a job into the queue himself and hence causing the job to be executed with malicious code. I was told that I could include a password inside the job being sent.
Though after some time passed, I recognized that someone could preform a few simple commands on a terminal and obtain the job inside the queue, and hence find the password, and then create jobs with the password included:
telnet thewebsitesipaddress 11300 //creating a telnet connection
list-tubes //finding which tubes are currently being used
use a_tube_found //using one of the tubes found
peek-ready //see whats inside one of the jobs and find the password
What could be done to make sure this does not happen and my queue doesn't get hacked / controlled?
Thanks in advance!
You can avoid those situations by placing beanstalkd behind a firewall or in a private network.
DigitalOcean (for example) offers such a service where you have a private network IP address which can be accessed only from servers of the same location.
We've been using beanstalkd in our company for more than a year, and we haven't had any of those issues yet.
I see, but what if the producer was a page called index.php, where when someone entered it, a job would be sent to the queue. In this situation, wouldn't the server have to be an open network?
The browser has no way to get in contact with the job server, it only access the resources /you/ allow them to, that is the view page. Only the back-end is allowed to access the job server. Also, if you build the web application in a certain way that the front-end is separated from the back-end, you're going to have even less potential security issues.
current setup,
2 master servers, 12 worker servers:
workers are connected to master through ssh-copy-id, masters and workers are writing data in redis-queues on masters.
issue i have been facing for past week is that redis is writing data in the authorized_keys file, i cant reproduce this issue or confirm which server is doing this.
I looked into the redis config file and i didn't find any setting that would make redis write in authorized_keys file.
Has anyone else faced this issue or similar, i clear the authorized keys file and it writes into it again.
Your servers are most probably being/have been attacked by a "cracker". While it is possible that attack is over, you should treat your servers as compromised and act accordingly. This is in all likelihood the same approach described by Salvatore Sanfilippo a.k.a antirez, Redis' author and security researcher in his past, in this blog post.
To prevent this type of attacks which use Redis as a vector, please refer to the instructions in the Securing Redis in the Quicktart page as a starting point and the Security page for more information.
More discussion is at /r/redis
Update: more ramblings on the same topic at https://redislabs.com/blog/3-critical-points-about-security
I wrote a multi-process realtime WebSocket server which uses the session id to load-balance traffic to the relevant worker based on the port number that it is listening on. The session id contains the hostname, source port number, worker port number and the actual hash id which the worker uses to uniquely identify the client. A typical session id would look like this:
localhost_9100_8000_0_AoT_eIwV0w4HQz_nAAAV
I would like to know the security implications for having the worker port number (in this case 9100) as part of the session id like that.
I am a bit worried about Denial of Service (DoS) threats - In theory, this could allow a malicious user to generate a large number of HTTP requests targeted at a specific port number (for example by using a fake sessionID which contains that port number) - But is this a serious threat? (assuming you have decent firewalls)? How do big companies like Google handle dealing with sticky sessions from a security perspective?
Are there any other threats which I should consider?
The reason why I designed the server like this is to account for the initial HTTP handshake and also for when the client does not support WebSocket (in which case HTTP long-polling is used - And hence subsequent HTTP requests from a client need to go to the same worker in the backend).
So there are several sub-questions in your question. I'll try to split them up and answer them accordingly:
Is DoS-Attack on a specific worker a serious threat?
It depends. If you will have 100 users, probably not. But you can be sure, that there are people out there, which will have a look at your application and will try to figure out the weaknesses and exploit those.
Now is a DoS-Attack on single workers a serious possibility, if you can just attack the whole server? I would actually say yes, because it is a more precise attack => you need less resources to kill the workers when you do it one by one. However if you allow connection from the outside only on port 80 for HTTP and block everything else, this problem will be solved.
How do big companies like Google handle dealing with sticky sessions?
Simple answer - who says, they do? There are multiple other ways to solve the problem of sessions, when you have a distributed system:
don't store anything session based on the server, just have a key in the cooky with which you can identify the user again, similar as with automatic login.
store the session state in a data base or object storage (this will add a lot of overhead)
store session information in the proxy (or broker, http endpoint, ...) and send them together with the request to the next worker
Are there any other threats which I should consider?
There are always unforeseen threats, and that's the reason, why you should never publish more information than necessary. In that case, most big companies don't even publish the correct name and version of their WebServer (for google it is gws for instance)
That being said, I see your point why you might want to keep your implementation, but maybe you can modify it slightly to store in your load balancer a dictionary with a hashed value of hostname, source port number, worker port number and have as a session id a collection of two hashes. Than the load balancer knows, by looking into the dictionary, to which worker it needs to be sent. This info should be saved together with a timestamp, when the info was retrieved the last time, and every minute you can delete unused data.
Hi I am setting up a cluster of machines using chef at offsite locations. If one of these machines was stolen, what damage can the attacker do to my chef-server or other nodes by having possession of chef-validator.pem ? What other things can they access through chef? Thanks!
This was one of the items discussed at a recent Foodfight episode on managing "secrets" in chef. Highly recommended watching:
http://foodfightshow.org/2013/07/secret-chef.html
The knife bootstrap operation uploads this key when initializing new chef clients. Possession of this key enables the client to register itself against your chef server. That is actually its only function, once the client is up and running the validation key is no longer needed.
But it can be abused.... As #cbl has pointed out, if an unauthorized 3rd party gets access to this key they can create new clients that can see everything on your chef server that normal clients can see. It can theoretically be used to create a Denial of Service attack on your chef server, by flooding it with registration requests.
The foodfight panel recommend a simple solution. Enable the chef-client cookbook on all nodes. It contains a "delete_validation" recipe that will remove the validation key and reduce your risk exposure.
The validator key is used to create new clients on the Chef Server.
Once the attacker gets hold of it, he can pretend he's a node in your infrastructure and have access to the same information any node has.
If you have sensitive information in an unencrypted data bag, for example, he'll have access to that.
Basically he'll be able to run any recipe from any cookbook, do searches (and have access to all your other nodes' attributes), read data bags, etc.
Keep that in mind when writing cookbooks and populating the other objects in the server. You could also somehow monitor the chef server for any suspicious client creation activity, and if you have any reason believe that the validator key has been stolen, revoke it and issue a new one.
It's probably a good idea to rotate the key periodically as well.
As of Chef 12.2.0 the validation key is no longer required:
https://blog.chef.io/2015/04/16/validatorless-bootstraps/
You can delete your validation key on your workstation and then knife will use your user credentials to create the node and client.
There's also some other nice features of this since whatever you supply for the run_list and environment is also applied to the node when it is created. No more relying on the first-boot.json file to be read by the chef-client and the run having to complete before the node.save creates the node at the end of the bootstrapping process.
Basically, chef-client uses 2 mode authentication for to the server :-
1) organization validator.pem and
2) user.pem
Unless and until there is the correct combination of these 2 keys. chef-client wont be able to authenticate with the chef server.
They can even connect any node to the chef server with the stolen key via the following steps.
Copying and pasting the validator key into /etc/chef folder on any machine
Creating client.rb file with the following details
log_location STDOUT
chef_server_url "https://api.chef.io/organizations/ORGNAME"
validation_client_name 'ORGNAME-validator'
validation_key '/etc/chef/validater.pem'
3: Run chef-client to connect to the chef server