We've hired a security consultant to perform a pentest on our Application's public IP (Kubernetes Loadbalancer) and write a report on our security flaws and the measurements required to avoid them. Their report warned us that we have TCP Timestamp enabled, and from what I've read about the issue, It would allow an attacker to predict boot time of the machine thus being able to grant control over it.
I also read that TCP Timestamp is important for TCP performance and, most importantly, for Protection Against Wrapping Sequence.
But since we use Kubernetes over GKE with Nginx Ingress Controller being in front of it, I wonder if that TCP Timestamp thing really matters for that context. Should we even care? If so, does it really make my network vulnerable for the lack of Protection Against Wrapping sequence?
More information about TCP Timestamp on this other question:
What benefit is conferred by TCP timestamp?
According to RFC 1323 (TCP Extensions for High Performance) TCP Timestamp is used for two main mechanisms:
PAWS (Protect Against Wrapped Sequence)
RTT (Round Trip Time)
PAWS - defense mechanism for identification and rejection of packets that arrived in other wrapping sequence (data integrity).
Round Trip Time - time for packet to get to the destination and sent acknowledgment back to the device it originated.
What can happen when you disable TCP Timestamps:
Turning off TCP Timestamp can result with performance issues because the RTT would stop working.
It will disable PAWS.
As McAfee site says disabling timestamps can allow denial attacks.
As previously mentioned McAfee's site:
For these reasons, McAfee strongly recommends keeping this feature enabled and considers the vulnerability as low..
-- McAfee
Citation from another site:
Vulnerabilities in TCP Timestamps Retrieval is a Low risk vulnerability that is one of the most frequently found on networks around the world. This issue has been around since at least 1990 but has proven either difficult to detect, difficult to resolve or prone to being overlooked entirely.
-- Beyond Security
I would encourage you to look on this video: HIP15-TALK:Exploiting TCP Timestamps.
What about GKE
Getting the information about boot time (uptime in this case) can lead to knowledge about what security patches are not applied to the cluster. It can lead to exploitation of those unpatched vulnerabilities.
The best way to approach that would be regularly update existing cluster.
GKE implements 2 ways of doing that:
Manual way
Automatic way
Even if attacker knows the boot time of your machine it will be useless because system is up to date and all the security patches are applied.
There is dedicated site for Kubernetes engine security bulletins: Security bulletins
Related
I've found other answers to my question on StackOverflow, but they are old, and things have changed a lot when it comes to security.
I would like a year 2017 re-evaluation of the answer to this question:
Inside my company, should my database require TLS connections from our internal client applications? My database has customer data that would constitute a Yahoo-sized public relations disaster if it leaked. But my client applications are firewall'd away from any public internet access.
TLS is used to prevent someone from sniffing traffic. But from my understanding (correct me if I'm wrong) the only way sniffing traffic is possible is to be a root user on the source or destination server. (assuming we don't have shared Ethernet hubs and such)
The may be be another way to sniff traffic if someone has direct physical access to the network switches in our datacenter.
But trying to protect ourselves against root users and datacenter employees seems overkill to me, especially given that TLS does come at a considerable cost in terms of performance, complexity, and maintainability.
Or is this just one of those situations where we need to be absolute in our security choices rather than try to reason cost-benefit, because cost-benefit is impossible to calculate accurately when it comes to security breaches anyway?
Thanks! :)
Using TLS for an internal network helps simplify the threat model. More would have to go wrong in order for the transmitted data to be leaked to an adversary.
An adversary who has compromised any system on the same network as the database can use ARP spoofing to reroute traffic. From a threat modeling perspective, this increases the attack surface, and every system on the same network needs to have the same security guarantee. Many routers have a configuration option that protects the ARP table from such attacks - and this option can be disabled by a firmware update. Using TLS is defense-in-depth because this measure protects core assets, even when other systems fail.
Watched a CBT Nugget video and it was said that BGP protocol was slow. So if you brought up a domain it would take days for the domain to be fully accessible. However while at work a change was made on the router concerning BGP routes and it took minutes for the change to seen. So is the BGP protocol slow or is it fast. Thanks
It’s common knowledge that BGP has no ability to make performance-based routing decisions and often routes traffic through paths that are congested or affected by routing anomalies.
Since BGP is focused on reachability and its own stability, in case some problems occur the traffic may only be rerouted due to hard failures. Hard failures are total losses of reachability as opposed to degradation. This means that even though service may be so degraded that it is unusable for an end user, BGP will continue to assume that a degraded route is valid until and unless the route is invalidated by a total lack of reachability.
One way to detect this problem is to monitor reachability of key remote services.
Another is looking at total traffic, which will be much lower than usual in the presence of a black hole. Once detected, recovering from routing black hole affecting one ISP/peer is very SIMPLE: shut down the BGP session towards that provider until they’ve fixed the problem.
Alternatively you can automate the process of selecting the best performing transit provider or peer by deploying a route optimizer which evaluates all ISPs, IXes, and partial peers in terms of packet loss and latency and automatically reroutes traffic through the most reliable path.
So, BGP can be fast if correctly configured and optimized.
Considering the recent changes done in few minutes as you mention, It actually while working with the ISPs, it depends on how large their network is and based on their topologies they have to make a decision based on the AS. The larger the AS the more time it will take. By larger, I meant the hops and advertisements and network Segments. While working on Simulation environment it takes around 30seconds. But the production network is different.
In general it is about the definition of "fast".
Compared to IGPs, BGP is really slow.
Inside a production network where you may need subsecond reconvergence of your network, that is very fast compared to some minutes (iBGP is not discussed here, because this can be highly optimized (IGPs/BFD/Timers).
eBGP can't because you lose control at the border of your AS). Your delay gets bigger with every hop (maybe the next provider) your route has to be propagated. You might have seen your change after minutes, but that does not mean, that this change was propagated around the globe.
You can check the changes here with some kind of history:
https://stat.ripe.net/special/bgplay
I can appreciate that seeing "basic auth" and "safe enough" in the same sentence is a lot like reading "Is parachuting without a parachute still safe?", so I'll do my best to clarify what I am getting at.
From what I've seen online, people typically describe basic HTTP auth as being unsecured due to the credentials being passed in plain text from the client to the server; this leaves you open to having your credentials sniffed by a nefarious person or man-in-the-middle in a network configuration where your traffic may be passing through an untrusted point of access (e.g. an open AP at a coffee shop).
To keep the conversation between you and the server secure, the solution is to typically use an SSL-based connection, where your credentials might be sent in plain text, but the communication channel between you and the server is itself secured.
So, onto my question...
In the situation of replicating one CouchDB instance from an EC2 instance in one region (e.g. us-west) to another CouchDB instance in another region (e.g. singapore) the network traffic will be traveling across a path of what I would consider "trusted" backbone servers.
Given that (assuming I am not replicating highly sensitive data) would anyone/everyone consider basic HTTP auth for CouchDB replication sufficiently secure?
If not, please clarify what scenarios I am missing here that would make this setup unacceptable. I do understand for sensitive data this is not appropriate, I just want to better understand the ins and outs for non-sensitive data replicated over a relatively-trusted network.
Bob is right, it is better to err on the side of caution, but I disagree. Bob could be right in this case (see details below), but the problem with his general approach is that it ignores the cost of paranoia. It leaves "peace dividend" money on the table. I prefer Bruce Schneier's assessment that it is a trade-off.
Short answer
Start replicating now! Do not worry about HTTPS.
The greatest risk is not wire sniffing, but your own human error, followed by software bugs, which could destroy or corrupt your data. Make a replica!. If you will replicate regularly, plan to move to HTTPS or something equivalent (SSH tunnel, stunnel, VPN).
Rationale
Is HTTPS is easy with CouchDB 1.1? It is as easy as HTTPS can possibly be, or in other words, no, it is not easy.
You have to make an SSL key pair, purchase a certificate or run your own certificate authority—you're not foolish enough to self-sign, of course! The user's hashed password is plainly visible from your remote couch! To protect against cracking, will you implement bi-directional SSL authentication? Does CouchDB support that? Maybe you need a VPN instead? What about the security of your key files? Don't check them into Subversion! And don't bundle them into your EC2 AMI! That defeats the purpose. You have to keep them separate and safe. When you deploy or restore from backup, copy them manually. Also, password-protect them so if somebody gets the files, they can't steal (or worse, modify!) your data. When you start CouchDB or replicate, you must manually input the password before replication will work.
In a nutshell, every security decision has a cost.
A similar question is, "should I lock my house at night? It depends. Your profile says you are in Tuscon, so you know that some neighborhoods are safe, while others are not. Yes, it is always safer to always lock all of your doors all of the time. But what is the cost to your time and mental health? The analogy breaks down a bit because time invested in worst-case security preparedness is much greater than twisting a bolt lock.
Amazon EC2 is a moderately safe neighborhood. The major risks are opportunistic, broad-spectrum scans for common errors. Basically, organized crime is scanning for common SSH accounts and web apps like Wordpress, so they can a credit card or other database.
You are a small fish in a gigantic ocean. Nobody cares about you specifically. Unless you are specifically targeted by a government or organized crime, or somebody with resources and motivation (hey, it's CouchDB—that happens!), then it's inefficient to worry about the boogeyman. Your adversaries are casting broad nets to get the biggest catch. Nobody is trying to spear-fish you.
I look at it like high-school integral calculus: measuring the area under the curve. Time goes to the right (x-axis). Risky behavior goes up (y-axis). When you do something risky you saved time and effort, but the the graph spikes upward. When you do something the safe way, it costs time and effort, but the graph moves down. Your goal is to minimize the long-term area under the curve, but each decision is case-by-case. Every day, most Americans ride in automobiles: the single most risky behavior in American life. We intuitively understand the risk-benefit trade-off. Activity on the Internet is the same.
As you imply, basic authentication without transport layer security is 100% insecure. Anyone on EC2 that can sniff your packets can see your password. Assuming that no one can is a mistake.
In CouchDB 1.1, you can enable native SSL. In earlier version, use stunnel. Adding SSL/TLS protection is so simple that there's really no excuse not to.
I just found this statement from Amazon which may help anyone trying to understand the risk of packet sniffing on EC2.
Packet sniffing by other tenants: It is not possible for a virtual instance running in promiscuous mode to receive or "sniff" traffic that is intended for a different virtual instance. While customers can place their interfaces into promiscuous mode, the hypervisor will not deliver any traffic to them that is not addressed to them. This includes two virtual instances that are owned by the same customer, even if they are located on the same physical host. Attacks such as ARP cache poisoning do not work within EC2. While Amazon EC2 does provide ample protection against one customer inadvertently or maliciously attempting to view another's data, as a standard practice customers should encrypt sensitive traffic.
http://aws.amazon.com/articles/1697
I’m looking at creating a P2P system. During initial research, I’m reading from Peer-to-Peer – Harnessing the Power of Disruptive Technologies. That book states “a fully decentralized approach to instant messaging would not work on today's Internet.” Mostly blaming firewalls and NATs. The copyright is 2001. Is this information old or still correct?
It's still largely correct. Most users still are behind firewalls or home routers that block incoming connections. Those can be opened easier today than in 2001 (using uPnP for example, requiring little user interaction and knowledge) but most commercial end-user-targeting applications - phone (Skype, VoIP), chat (the various Messengers), remote control - are centralized solutions to circumvent firewall problems.
I would say that it is just plain wrong, both now and then. Yes, you will have many nodes that will be firewalled, however, you will also have a significant number who are not. So, if end-to-end encryption is used to protect the traffic from snooping, then you can use non-firewalled clients to act as intermediaries between two firewalled clients that want to chat.
You will need to take care, however, to spread the load around, so that a few unfirewalled clients aren't given too much load.
Skype uses a similar idea. They even allow file transfers through intermediaries, though they limit the through-put so as not to over load the middle-men.
That being said, now in 2010, it is a lot easier to punch holes in firewalls than it was in 2001, as most routers will allow you to automate the opening of ports via UPNP, so you are likely to have a larger pool of unfirewalled clients to work with.
Firewalls and NATs still commonly disrupt direct peer-to-peer communication between home-based PCs (and also between home-based PCs and corporate desktops).
They can be configured to allow particular peer-to-peer protocols, but that remains a stumbling block for most unsavvy users.
I think the original statement is no longer correct. But the field of Decentralized Computing is still in its infancy, with little serious contenders.
Read this interesting post on ZeroTier (thanks to #joehand): The State of NAT Traversal:
NAT is Traversable
In reading the Internet chatter on this subject I've been shocked by how many people don't really understand this, hence the reason this post was written. Lots of people think NAT is a show-stopper for peer to peer communication, but it isn't. More than 90% of NATs can be traversed, with most being traversable in reliable and deterministic ways.
At the end of the day anywhere from 4% (our numbers) to 8% (an older number from Google) of all traffic over a peer to peer network must be relayed to provide reliable service. Providing relaying for that small a number is fairly inexpensive, making reliable and scalable P2P networking that always works quite achievable.
I personally know of Dat Project, a decentralized data sharing toolkit (based on their hypercore protocol for P2P streaming).
From their Dat - Distributed Dataset Synchronization And Versioning paper:
Peer Connections
After the discovery phase, Dat should have a list of
potential data sources to try and contact. Dat uses
either TCP, UTP, or HTTP. UTP is designed to not
take up all available bandwidth on a network (e.g. so
that other people sharing wifi can still use the Inter-
net), and is still based on UDP so works with NAT
traversal techniques like UDP hole punching.
HTTP is supported for compatibility with static file servers and
web browser clients. Note that these are the protocols
we support in the reference Dat implementation, but
the Dat protocol itself is transport agnostic.
Furthermore you can use it with Bittorrent DHT. The paper also contains some references to other technologies that inspired Dat.
For implementation of peer discovery, see: discovery-channel
Then there is also IPFS, or 'The Interplanetary File System' which is currently best positioned to become a standard.
They have extensive documentation on their use of DHT and NAT traversal to achieve decentralized P2P.
The session messenger seem to have solved the issue with a truly decentralized p2p messenger by using a incentivized mixnet to relay and store messages. Its a fork of the Signal messenger with a mixnet added in. https://getsession.org -- whitepaper: https://getsession.org/wp-content/uploads/2020/02/Session-Whitepaper.pdf
It's very old and not correct. I believe there is a product out called Tribler (news article) which enables BitTorrent to function in a fully decentralized way.
If you want to go back a few years (even before that document) you could look at Windows. Windows networking used to function in a fully decentralized way. In some cases it still does.
UPNP is also decentralized in how it determines available devices on your local network.
In order to be decentralized you need to have a way to locate other peers. This can be done proactively by scanning the network (time consuming) or by having some means of the clients announcing that they are available.
The announcements can be simple UDP packets that get broadcast every so often to the subnet which other peers listen for. Another mechanism is broadcasting to IIRC channels (most common for command and control of botnets), etc. You might even use twitter or similar services. Use your imagination here.
Firewalls don't really play a part because they almost always leave open a few ports, such as 80 (http). Obviously you couldn't browse the network if that was closed. Now if the firewall is configured to only allow connections that originated from internal clients, then you'd have a little more work to do. But not much.
NATs are also not a concern for similiar issues.
There are limits imposed by available memory, bandwidth, CPU, and of course, the network connectivity. But those can often be scaled vertically. Are there any other limiting factors on linux? Can they be overcome without kernel modifications? I suspect that, if nothing else, the limiting factor would become the gigabit ethernet. But for efficient protocols it could take 50K concurrent connections to swamp that. Would something else break before I could get that high?
I'm thinking that I want a software udp and/or tcp/ip load balancer. Unfortunately nothing like that in the open-source community seems to exist, except for the http protocol. But it is not beyond my abilities to write one using epoll. I expect it would go through a lot of tweaking to get it to scale, but that's work that can be done incrementally, and I would be a better programmer for it.
The one parameter you will probably have some difficulty with is jitter. Has you scale the number of connections per box, you will undoubtedly put strain on all the resources of the said system. As a result, the jitter characteristics of the forwarding function will likely suffer.
Depending on your target requirements, that might or not be an issue: if you plan to support mainly elastic traffic (traffic which does not suffer much from jitter and latency) then it's ok. If the proportion of inelastic traffic is high (e.g. interactive voice/video), then this might be more of an issue.
Of course you can always over engineer in this case ;-)
If you intend to have a server which holds one socket open per client, then it needs to be designed carefully so that it can efficiently check for incoming data from 10k+ clients. This is known as the 10k problem.
Modern Linux kernels can handle a lot more than 10k connections, generally at least 100k. You may need some tuning, particularly the many TCP timeouts (if using TCP) to avoid closing / stale sockets using up lots of resource if a lot of clients connect and disconnect frequently.
If you are using netfilter's conntrack module, that may also need tuning to track that many connections (this is independent of tcp/udp sockets).
There are lots of technologies for load balancing, the most well-known is LVS (Linux Virtual Server) which can act as the front end to a cluster of a real servers. I don't know how many connections it can handle, but I think we use it with at least 50k in production.
To your question, you are only restrained by hardware limitations. This was the design philosophy for linux systems. You are describe exactly what would be your limiting factors.
Try HAProxy software load balancer:
http://haproxy.1wt.eu/