Assume the following scenario:
I have two different servers, A and B.
Server A hosts a static file, representing a set of valid identifiers. Let's say those are represented as integers. The set is rather large; assume it contains around 30-50% of all 4 byte integers. The contents of this file never changes.
Clients can download this set from server A. Clients must be able to transparently see the actual int values returned by A.
Clients can then invoke some API function on server B, passing in some of those identifiers (int values) to server B
Server B must verify that the passed-in integer is "valid", meaning it is indeed part of the set of integers hosted on server A
Server B has no ongoing access to server A, and it has no knowledge of the static data file
Is there a secure way for server B to verify the parameter passed in from the client?
My idea so far would be:
Server A exposes the numbers, as well as an encrypted version of the number (encrypted with, say, AES256)
The key used for encryption is shared between server A and server B
Clients can call host B using the encrypted number
Host B can decrypt this and therefore verify the number originates from host A, and thereby must have been part of the set of numbers
Would this approach be secure?
Given the exposed set of integers covers a large fraction of all possible integer values, and clients can see the encrypted version of each of those - does that open up any way to reverse engineer the encryption key? Or suffer from any other security flaws?
Why am I thinking of doing something dodgy like this at all? In short, cost reduction.
I'm exposing a very large, chunked dataset that clients can download in parts (or - in case of any malicious actors - in full). They need to be able to invoke a server API requesting do invoke certain actions based on this data. The server needs to ensure the input data is valid.
The natural approach of course would be for the host to load up the data from internal sources to verify. Since this is all going to be hosted on AWS serverless, that would mean extra cost for each request though (reading a file, or reading a DB, etc. - all charged separately), which I'd ideally like to avoid. If I could trust the data passed in by the client, I might be able to dodge some costs and cut a little latency while at it.
Related
I have an API endpoint, which creates and sends a few transactions in strict sequence. Because I don't wait for results of these transactions, I specify a nonce number for each of them to execute them in the right order.
This endpoint is built using AWS Lambda function. So, if I have many concurrent requests, the lambda runs in concurrent mode. In this case, several concurrent instances can get the same nonce (I'm using eth.getTransactionCount method to get the latest transaction count) and send a few transactions with the same nonce. Therefore, I receive errors because instead of creating a new transaction, it tries to replace an existing one.
Basically, I need a way to check if a nonce is already taken right before the transaction sending or somehow reserve a nonce number (is it even possible?).
The web3 getTransactionCount() only returns the amount of already mined transactions, but there's currently no way to return the highest pending nonce (for an address) using web3.
So you'll need to store your pending nonces in a separate DB (e.g. Redis). Each Lambda run will need to access this DB to get the highest pending nonce, calculate one that it's going to be using (probably just +1), and store this number to the DB so that other instances can't use it anymore.
Mind that it's recommended to implement a lock (Redis, DynamoDB) to prevent multiple app instances from accessing the DB and claiming the same value at the same time.
Basically, I need a way to check if a nonce is already taken right before the transaction sending or somehow reserve a nonce number (is it even possible?).
You should not.
Instead, you should manage nonce in your internal database (SQL, etc.) which provides atomic counters and multiple readers and writers. You only rely to the network provided nonce if 1) your system has failed 2) you need to manually reset it.
Here is an example code for Web3.py and SQLAlchemy.
I'm wondering, how modern DNS servers dealing with millions queries per second, due to the fact that txnid field is uint16 type?
Let me explain. There is intermediate server, from one side clients sending to it DNS requests, and from other side server itself sending requests to upper DNS server (8.8.8.8 for example). So the thing is, that according to DNS protocol there is field txnid in the DNS header, which should be unchanged during request and response. Obviously, that intermediate DNS server with multiple clients replace this value with it's own txnid value (which is a counter), then sends request to external DNS server and after resolving replace this value back to client's one. And all of this will work fine for 65535 simultaneous requests due to uint16 field type. But what if we have hundreds of millions of them like Google DNS servers?
Going from your Google DNS server example:
In mid-2018 their servers were handling 1.2 trillion queries-per-day, extrapolating that growth says their service is currently handling ~20 million queries-per-second
They say that successful resolution of a cache-miss takes ~130ms, but taking timeouts into account pushes the average time up to ~400ms
I can't find any numbers on what their cache-hit rates are like, but I'd assume it's more than 90%. And presumably it increases with the popularity of their service
Putting the above together (2e7 * 0.4 * (1-0.9)) we get ~1M transactions active at any one time. So you have to find at least 20 bits of state somewhere. 16 bits comes for free because of the txnid field. As Steffen points out you can also use port numbers, which might give you another ~15 bits of state. Just these two sources give you more than enough state to run something orders of magnitude bigger than Google's DNS system.
That said, you could also just relegate transaction IDs to preventing any cache-poisoning attacks, i.e. reject any answers where the txnid doesn't match the inflight query for that question. If this check passes, then add the answer to the cache and resume any waiting clients.
This requires a little context, so bear with me.
Suppose you're building a chat app atop CouchDB that functioned like IRC (or Slack). There's a server and some clients. But in this case, the server and the clients each have a CouchDB database and they're all bidirectionally replicating to each other -- the clients to the server, and the server to the other clients (hub-and-spoke style). Clients send messages by writing to their local instance, which then replicates to the server and out to the other clients.
Is there any way (validation function?) to prevent hostile clients from inserting a billion records and replicating those changes up to the server and other clients? Or is it a rule that you just can't give untrusted clients write access to a CouchDB instance that replicates anywhere else?
Related:
couchdb validation based on content from existing documents
Can I query views from a couchdb update or validate_doc_update function?
Can local documents be disabled in CouchDB?
For a rather simple defense agaist flooding, I am using the following workflow:
All public write access is only allowed through update functions
Every document insert/update gets generated a unique hash, consisting of the req.peer field (for the IP address) and an ISO timestamp where I cut off the final part. For example I may have 2017-11-24T14:14 as they key unique string, so that ensures that a unique key is generated every minute
Calculate the hash for every write request, ensure it is unique, and you will be certain a given IP would only be allowed to write once every minute.
This technique works ok for small floods, coming from a given set of IPs. For a more coordinated attack a variation (or even something else completely) might be needed.
I have a public computer that is used in an ATM sort of fashion. When a certain action occurs (person inserts money), the program I've written on the computer sends a request to a trusted server which does a very critical task (transfers money).
I'm wondering, since I have to communicate to a server to start the critical task, the credentials to communicate with it are stored on this public computer. How do I prevent hackers from obtaining this information and running the critical task with their own parameters?
HSM (Hardware Security Modules) are designed to store keys safely:
A hardware security module (HSM) is a physical computing device that safeguards and manages digital keys for strong authentication and provides cryptoprocessing. These modules traditionally come in the form of a plug-in card or an external device that attaches directly to a computer or network server.
HSMs may possess controls that provide tamper evidence such as logging and alerting and tamper resistance such as deleting keys upon tamper detection. Each module contains one or more secure cryptoprocessor chips to prevent tampering and bus probing.
Impossible in general
If your user has access to this PC, they can easily insert fake money. Your model is doomed.
Minimize attack surface
This PC ought to have unique token (a permanent cookie is enough), and sever will refuse a request without a valid cookie. Server maintains database of device types, and this ATM-PC is only allowed certain operations (deposit money up to NNN units). Ideally it is also rate-limited (at most once per 3 seconds).
I wrote a multi-process realtime WebSocket server which uses the session id to load-balance traffic to the relevant worker based on the port number that it is listening on. The session id contains the hostname, source port number, worker port number and the actual hash id which the worker uses to uniquely identify the client. A typical session id would look like this:
localhost_9100_8000_0_AoT_eIwV0w4HQz_nAAAV
I would like to know the security implications for having the worker port number (in this case 9100) as part of the session id like that.
I am a bit worried about Denial of Service (DoS) threats - In theory, this could allow a malicious user to generate a large number of HTTP requests targeted at a specific port number (for example by using a fake sessionID which contains that port number) - But is this a serious threat? (assuming you have decent firewalls)? How do big companies like Google handle dealing with sticky sessions from a security perspective?
Are there any other threats which I should consider?
The reason why I designed the server like this is to account for the initial HTTP handshake and also for when the client does not support WebSocket (in which case HTTP long-polling is used - And hence subsequent HTTP requests from a client need to go to the same worker in the backend).
So there are several sub-questions in your question. I'll try to split them up and answer them accordingly:
Is DoS-Attack on a specific worker a serious threat?
It depends. If you will have 100 users, probably not. But you can be sure, that there are people out there, which will have a look at your application and will try to figure out the weaknesses and exploit those.
Now is a DoS-Attack on single workers a serious possibility, if you can just attack the whole server? I would actually say yes, because it is a more precise attack => you need less resources to kill the workers when you do it one by one. However if you allow connection from the outside only on port 80 for HTTP and block everything else, this problem will be solved.
How do big companies like Google handle dealing with sticky sessions?
Simple answer - who says, they do? There are multiple other ways to solve the problem of sessions, when you have a distributed system:
don't store anything session based on the server, just have a key in the cooky with which you can identify the user again, similar as with automatic login.
store the session state in a data base or object storage (this will add a lot of overhead)
store session information in the proxy (or broker, http endpoint, ...) and send them together with the request to the next worker
Are there any other threats which I should consider?
There are always unforeseen threats, and that's the reason, why you should never publish more information than necessary. In that case, most big companies don't even publish the correct name and version of their WebServer (for google it is gws for instance)
That being said, I see your point why you might want to keep your implementation, but maybe you can modify it slightly to store in your load balancer a dictionary with a hashed value of hostname, source port number, worker port number and have as a session id a collection of two hashes. Than the load balancer knows, by looking into the dictionary, to which worker it needs to be sent. This info should be saved together with a timestamp, when the info was retrieved the last time, and every minute you can delete unused data.