Can two or more domains be hosted on the same server? If yes what is the ip address we are going to get for the two domains?
as a user can i know how a server resolves the host name and assign unique id to different host names
After looking at my comment above and as you are a user, not an administrator, just look at the documentation of nslookup(1). It is a tool to make DNS queries to servers. It allow you to make dns resolution and to investigate the ways you are getting the answer (there are many ways to answer a query, believe me)
First you need to know how the asking is being done. Normally, clients make recursive queries (they want definitive answers to a query and want the server to do the heavy work) and servers do iterative ones (they approximate the answer by asking the servers in the chain to the final domain you are looking for) Servers and clients normally cache results for future questions and provide several ways of fault tolerance, so you cannot control normally how a query is solved. As this is probably the most requested service in internet, the protocol has been optimized to get quick answers even in the worst case.
Once you get an answer, it can be a partial one, it can be cached, it can be non-authoritative (meaning the server is serving a cached entry, not a locally administered one)
When you have several responses to a query (ok, this can happen) you receive them normally in order, depending where are you querying from. The server makes a best effort to order them on proximity to the client (the nearest address is served first) and/or randomly ordered, so you can make round robing to each of the addresses you receive. It depends on the client software, the server implementation, the administrator policy, etc.
Even you can receive a different response depending on who you are. Several corporate servers serve different views of the database depending on where the clients come from. If they come from the inside of the company, they serve addresses for servers not visible from the outside. For example, if you try to access the corporate web server, you can receive the private address to reach it, not the public address of the server accesible from the internet. This concept is called view, and many servers implement it, so the answer to your question is: it depends :)
Related
I am writing to you because I have a conception problem for my DNS infrastructure.DNS. My infrastructure is composed of a DNS machine (recursive or forwarding) and another authoritatve that has say views according to the source of the client (we can assimilate it to Bind even if it is not the case). This Auhoritative machine should not be queried directly but must go through the other. To summarize here is the infrastructure:
> Client Location 1 Client Location 2 Client Location 3
> \ | /
> DNS Recursive ou Forwarding
> |
> DNS Authoratitve with 3 « views ».
I thought of different solutions to solve these problems :
Create different ports on the DNS Recursive (or Forwading), each port containing a DNS that would correspond to a view that would query the Authoritative DNS (and thus recognize the origin). But I find this solution rather ugly and that will quickly increase if the number of views increases.
Use the DNS extension : EDNS to forward the client network (but that seems pretty complicated).
I wanted to know if you have other solutions and if not what would be the best.
Thank you in advance !
The first solution does not seem really workable as there is nearly no way to change from the default DNS port in various end clients OS. You would instead need separate recursive nameservers on separate IP addresses and each client configured with the specific nameserver it needs to use.
The second solution can work, it is ECS the "EDNS Client Subnet" feature, described in RFC7871 and supported in various nameservers. See for example in Bind: https://www.isc.org/wp-content/uploads/2017/04/ecs.pages.pdf
Now are you really sure you need this setup or that this is the only way to achieve your goals? It is difficult to propose other ideas as you describe from the get go your solution but not really your problem initially nor your constraints.
For example, it may be solved in some cases by just configuring each client with a different domain search list. client1 would have client1.example.com as suffix, client2 would have client2.example.com and so on. Now, with only one standard recursive nameserver and one authoritative one for example.com without any kind of extension or complicated setup, when client1 attempts to resolve www it will (may) get a different reply than client2 also attempting to resolve www as the final two fully qualified domain name would be indeed different (www.client1.example.com vs www.client2.example.com), because of the different search lists. This of course depends a lot on what kind of applications are running on each client.
The use of simpler nameservers such as dnsmasq may also help, but again your space problem is not defined enough to be sure what to suggest.
Ive been reading alot about nameserver the last days. For our websites we want to optimize the waiting time of the visitors that is caused by our namserver. I will have some questions about IP Anycast and the general function of the DNS. Let me start by explaining what I understood the DNS works from user side:
User X wants to visit www.example.com, the following steps happen to get the IP address:
1.Step: User X sends request to the Nameserver of his ISP or nameserver by choice.(recursive nameserver)
2.Step: If the adress is not found, the recursive nameserver will send a request to one of the 13 root nameserver to get the nameserver for the .com TLD
3.Step: Query the .com nameserver to get the auhorative nameserver
4.Step: Query the auhorative nameserver to get the ip-address for www.example.com
First I realized that as a owner of a website you can only optimize Step number 4 and all other steps are not in our hands.
I came across IP Anycast nameserver (what is also used for the 13root nameservers) and totally understand the concept of distributed machines. But what I dont understand is where the decision logic, to which of the distributed machines the user will be send, according to his "position",is implemented? I mean when i buy an anycast nameserver, the logic should be implemented on the .com nameserver (Step 3), so that this nameserver decides to which machine of my anycast nameserver the user will be send.
For me thats really hard to understand and im asking myself if it really works that way? I hope someone can help me with these understanding questions.
Beside of that i found out, that another small method to gain some speed for the user, is to only use A Records and no CName Records anymore.
Are there some more ways to optimize a nameserver?
Thanks in advance!
Your question is not really related to programming, but more to operations, and is also a little too vague ("Are there some more ways to optimize a nameserver?").
But let us try to give you pointers.
User X wants to visit www.example.com, the following steps happen to get the IP address:
Your following description is then mostly correct. Note that at each step, by default until very recently, a recursive nameserver will send the whole name queried to each nameserver. Recently, QNAME Minimization appeared as a standard and now recursive nameservers can send to each authoritativ nameservers only the labels it neeeds to reply. This enhances privacy without changes to the protocol, but is not widespread today because some authoritative nameservers do not work correctly when queried that way.
As a domain name owner you can indeed only have an impact at the last step. But remember that recursive nameservers have caches, so the list of root nameservers as well as the list of .COM nameservers for example are so "hot" (so often needed) that they surely sit always in resolvers' caches, so basically step 1 and 2 happens do not happen often (at start when cache is empty typicaly).
I came across IP Anycast nameserver (what is also used for the 13root nameservers) and totally understand the concept of distributed machines. But what I dont understand is where the decision logic, to which of the distributed machines the user will be send, according to his "position",is implemented?
First things first, IP anycasting is not specific to the DNS, it is just hugely popular here because
it solves the load balancing/fail over problem that all big TLDs have
it works specially well with DNS over UDP which is a simple one query one reply protocol.
So any service can theoretically be anycasted. It means that a given IP address just appears at different locations in the world.
To summarize very broadly, Internet traffic between providers (AS numbers) is exchanged at peering points, where they interconnect and each provider says "I know about IP block 192.0.2.0/24, please send me all traffic for it", etc. for each blocks
(again this is a summary. Blocks are allocated by RIRs, and yes by default this is not very much authenticated so BGP hijacks happen when another provider also says "give me this traffic" when it shouldn't - and it happens because of malicious goals or just simple human errors).
For a normal (technical term: "unicast") IP address, only a single provider (AS) will announce it somewhere (technically: announce its block not just a single IP) and everything will be configured in such a way that wherever the start of the exchange is, for this single IP as destination, it will arrive at the exact same box.
On the contrary, for an anycast IP address, either a single provider or multiple ones (that is multiple Autonomous Systems) will announce this IP at various locations (peering points) in the globe. At each peering point, traffic for this IP will get taken by the provider announcing it there and then it will route this traffic to a specific server "nearby". Announcements of the same IP at peering point A and peering point B, will drive corresponding traffic on one side in datacentre X and the other for datacentre Y.
For the client, when everything works, it does not change anything, as long as all the replying servers react the same way to the same query. The client does not (and sometimes can not) even know the IP is anycasted or that it want to location X when another client doing the same thing will instead hit location Y.
So in short nameservers "decide" nothing in this regard. At each point of the DNS resolution, when they need to contact nameserver NS1 they know its IP address is IP1 and they just open an UDP (or sometimes TCP) connection to this IP, absolutely normally. It is the underlying IP and BGP protocols that will, if anycast is in action, make the response come from the appropriate "close" server.
Note that anycasting in this way, for DNS, achieve both:
fail-over : if one server dies, with appropriate monitoring, its provider withdraws its IP announcement, that is this local copy kind of disappear and the traffic will automatically (in order of seconds) shift to any other instance where the same IP is announced
load-balancing : rougly speaking, if you anycast one IP on 2 locations, each should receive 50% of traffic. It is not true in practice, and is very complicated (read: impossible) to predict or even monitor, because it all depends on the peering points, the agreements between the providers and various other policies (simple example: if you peer at two points where on first there is only one provider sending you trafic, and at other point you have 100 providers with whom you exchange traffic, then you may get more connections going to the second instance... except of course if single provider at first peering point is an ISP with millions of clients, where the other 100 providers are single small organizations...)
So, some nameservers are anycasted. Nowadays all the root ones are (but this was not true 16 months ago, see https://b.root-servers.org/news/2017/04/17/anycast.html as b.root-servers.org was the last one to board the anycast wagon) as well as all big TLDs, sometimes even with more that one "Anycast DNS providers".
For any domain name, you can get some providers that will give you a DNS service for it, based on a "cloud" of anycasted nameservers.
See for example:
https://www.pch.net/services/dns_anycast
https://www.netnod.se/dns/netnod-dns-services
https://dyn.com/dns/network-map/
http://www.cdns.net/anycast.html
https://www.rcodezero.at/en/home/
https://aws.amazon.com/route53/
https://cloud.google.com/dns/
and many others.
Now following on a totally different topic:
Beside of that i found out, that another small method to gain some speed for the user, is to only use A Records and no CName Records anymore.
This is not really something you gain things with, and CNAME records are useful in many other cases.
Again, you need to remember that there are caches.
So even if your configuration is:
www.mywebsite.example CNAME www.mywebsite.example.somegreatCDN.example
www.mywebsite.example.somegreatCDN.example A 192.0.2.128
it is true that this means on paper two DNS requests to finally be able to do an HTTP query, but in practice things will be cached (even more so today with big public open resolvers such as 1.1.1.1 or 8.8.8.8 or 9.9.9.9, that are anycasted too in fact), so the difference will be negligible (and only impacts the first time, never again until it is in cache) ... especially in the case of HTTP and everything that happens later that is opening, frequently, dozens of connections to download javscript source codes, CSS files, fonts, etc. that may be hosted elsewhere.
A lot of websites use CNAME records without negative impact. See www.amazon.com for example, right now:
;; ANSWER SECTION:
www.amazon.com. 730 IN CNAME www.cdn.amazon.com.
www.cdn.amazon.com. 11 IN CNAME d3ag4hukkh62yn.cloudfront.net.
d3ag4hukkh62yn.cloudfront.net. 11 IN A 54.239.172.122
You may however argue that some names will be more popular than others and hence stay longer in cache, which is certainly the case.
And finally:
Are there some more ways to optimize a nameserver?
Based on what? We touched various subjects above, all are compromises, you sacrifice something (it may be just "money") to gain something else (redundancy, etc.). There is no generic rule to declare when this compromise makes sense or not, it will depend a lot on your situation and what you are trying to do.
You are right, and should be congratulated about that, that you should invest some time around DNS setup, both for security and performance reasons. While a lot of money is often invested in huge HTTP setup to sustain various problems or spikes of activity (but even the best fail sometimes, see the recent Amazon Prime Day opening that was a gigantic failure), but often people forget about the DNS because it is on the infrastructure level so not well known nor understood (using UDP makes it already stand out from all other known protocols, as this is rare).
For example there is another completely different thing (it is orthogonal to anycasting, so it can work with or without it, the goals are different) that is related: "geo-DNS" means when a nameserver will reply differently based from where the client asks. This is meant to give, for example, a different IP for a webserver, one that is closer to client (so in that case the webserver itself is probably not anycasted). This is done by just looking at the source IP from the DNS packet, but it is often not good enough because the authoritative nameservers only see as source IP the one from the recursive nameserver and not the real end client one and nowadays with big open public recursive nameservers the location should be far off, so you also have a specific DNS option called EDNS Client Subnet that can be passed between recursive and authoritative nameservers so that they get the end client real IP address (in fact a block not a single IP for privacy reasons) and can act upon it.
Short answer is: you are right. The NameServers is where you can optimize and all "IP Anycast" products I have seen is just a NameServer setup that has a lot of locations.
They use the same system as the "root servers of the internet" but this does not mean that they have the same function. The IP Anycast is simply a method for multiple servers in different locations to serve the same IP address.
From WIKIPEDIA (http://en.wikipedia.org/wiki/Anycast)
On the Internet, anycast is usually implemented by using Border Gateway Protocol to simultaneously announce the same destination IP address range from many different places on the Internet. This results in packets addressed to destination addresses in this range being routed to the "nearest" point on the net announcing the given destination IP address.
If you are using a big ISP like ASCIO or someone using ULTRADNS you probably do not have to worry about this step too much, but if the NS is a local ISP it is worth considering. Make sure you have NS where your visitors are.
I assume this is where you came into contact with "IP Anycast" products. None that I have seen offers anything to attack step 1-2-3 but rather offers a large setup of NameServers allowing them to reduce resolving time due to closeness of networks.
Let me know if you are of the understanding that the offer is for a root NameServer setup, because I would like to see this.
We use WebSockets to communicate with our EC2 instances.
Our script is served using nodejs and Express, and then initialize the WebSocket.
Right now ELB is used which makes life harder to identify the client IP.
Using x-forwarded-for header we can get the IP in the context of HTTP, but when it comes to WebSocket context in the server, it looks like it's not forwarded by Amazon.
We identified 2 options:
Communicate the WebSocket directly with the instance (using its public DNS).
Maintain some sort of sessionid, in which store the IP when in the context of HTTP and associate it with the sessionid. The client side will get its sessionid using the HTTP response, and will use it to on the WebSockets. The the server will be to identify the client and resolve its IP from the cache.
Both options are not great: 1 is not fault tolerant and 2 is complex.
Are there more solutions? Can Amazon somehow forward the IP? What is the best practice?
Thanks
I have worked with websockets and I have worked with ELB, but I've never worked with them together, so I didn't realize that an HTTP forwarder on an Elastic Load Balancer doesn't understand websocket requests...
So I take it you must be using a TCP forwarder, which explains why you're using a different port, and of course the TCP forwarder is protocol-unaware, so it won't be adding any headers at all.
An option that seems fairly generic and uncomplicated would be for the http side of your application to advise the websocket side by pushing the information across rather than storing it in a cache for retrieval. It's scalable and lightweight, assuming there's not an obstacle in your environment that makes it difficult or impossible to implement.
While generating the web page that loads the websocket, take the string "ipv4:" and the client's IP ("192.168.1.1," for example), concatenate and encrypt them, and make the result url-friendly:
/* pseudo-code */
base64_encode(aes_encrypt('ipv4:192.168.1.1','super_secret_key'))
Using that example key with 128 bit aes and that example IP address, I get:
/* actual value returned by pseudo-code above */
1v5n2ybJBozw9Vz5HY5EDvXzEkcz2A4h1TTE2nKJMPk=
Then when rendering the html for the page containing the websocket, dynamically build the url:
ws = new WebSocket('ws://example.com/sock?client=1v5n2ybJBozw9Vz5HY5EDvXzEkcz2A4h1TTE2nKJMPk=');
Assuming the querystring from the websocket is accessible to your code, you could base64_decode and then aes_decrypt the string found in the query parameter "client" using the super-secret key, and then verify that it begins with "ipv4:" ... if it doesn't, then it's not a legitimate value.
Of course, "ipv4:" (at the beginning of the string) and "client" (for the query parameter) were arbitrary choices and do not have any actual significance. My choice of 128 bit AES was also arbitrary.
The problem, of course, with this setup is that it is subject to a replay: a given client IP address will always generate the same value. If you only using the client IP address for "informational purposes" (such as logging or debugging) then this may be sufficient. If you are using it for anything more significant, you may want to expand this implementation -- for example, by adding a timestamp:
'ipv4:192.168.1.1;valid:1356885663;'
On the receiving end, decode the string and check the timestamp. If it is not +/- whatever interval in seconds that you deem safe, then don't trust it.
These suggestions all hinge on your ability to dynamically generate the websocket url, the browser's ability to connect with it, and you being able to access the querystring portion of the URL in the websocket request... but if those pieces will fall into place, maybe this will help.
Additional thoughts (from comments):
The timestamp I suggested, above, is seconds from the epoch, which gives you an incrementing counter that requires no statefulness in your platform -- it only requires that all of your server clocks are correct -- so it doesn't add unnecessary complexity. If the decrypted value contains a timestamp less than (for example) 5 seconds different (+/-) from the server's current time, then you know you're dealing with an authenticated client. The time interval permitted only needs to be as long as the maximum reasonable time for a client to attempt its websocket connection after loading the original page, plus the maximum skew of all your server clocks.
It is true, of course, that with NAT, multiple different users could be behind the same source IP address. It's also true, though far less less likely, that a user could actually make the websocket connection from a different source IP than the one where they originated the first http connection, and still be quite legitimate... and it sounds like the identity of the authenticate user may be a more important value for you than the actual source IP.
If you include the authenticated user's ID within the encrypted string as well, you have a value that is unique to origin IP, user account, and time, to a precision of 1 second. I think this is what you're referring to by additional salt. Adding the user account to the string should get you the information you're wanting.
'ipv4:192.168.1.1;valid:1356885663;memberid:32767;'
TLS should prevent discovery of the this encrypted string by an unauthorized party, but avoidance of replayability is also important because the generated URL is available in clear text in a user's browser's "view source" for the html page. You don't want a user who is authorized today but unauthorized tomorrow to be able to spoof their way in with a signed string that should be recognized as no longer valid. Keying to a timestamp and requiring it to fall in a very small valid window prevents this.
It depends on how serious the application is.
Basing any kind of decision on client IP address is a risky proposition. Basing security on it, even more so. While the suggestions offered so far work well within the given constraints, it would not be sufficient for a robust enterprise application.
Client IP addresses can be obscured by NATs, as already pointed out. So people accessing the Web from their place of work will often appear to have the same IP address. People's routers at home act as a NAT, so every family member at home accessing the Web will appear to the have the same IP address. Or even the same person accessing the application from a PC and a tablet...
Whether behind a NAT or not, using the application from two browsers on the same machine will appear to the have the same address. Similarly, multiple tabs in the same browser will appear to have the same address.
Other junction points like proxies or load balancers may also hide the original client IP address such that the thing behind the proxy/load balancers thinks they are the client. (More sophisticated or lower level intermediaries can prevent this, which is what makes them more sophisticated or expensive.)
Given all of the above, a serious application should not rely on client IP address for any kind of important decision, especially around security.
i dont want to do something illegal with it(e.g. vote continuously, in fact, somebody is doing it), but i only feel curious about it. For i have learned TCP/IP, and i found there are many software such like "IP changer",using which you can submit a website with different IP. WOW it is really magic! so i analysed some possible mechanism about it. But every possible way was denied by me.
i thought that they might connect and disconnect the internet continuously. because each time they connect the Internet, the ISP will dispatch a new IP address, and the hacker can make use of the new IP to submit the website, and disconnected after submitting successfully, and then connect for the next time...But it is impossible to some extent, for if do like this, every submitting will last a long time, and it doesn't work in some areas.
Modify TCP/IP data packets.For some time i did think it might be all right. but then i denied it. Assuming that i would submit a website, and i changed the IP address of the data packet which i will submit to the web site. it seems that everything is OK, but the web server will send message to the fake IP, so i wont get any information from the website. but in some circumstances where we only needn't reply it should work. Right? netfilter and iptables in linux may realize it, but i am not sure because i dont know the tools very well.
Using proxy server. i also think it is impossible to some extent.is there any method to get lots of free proxy servers? and most free proxy servers is very unstabitily, for there is a possible circumstance that you cannot use the proxy server in one day.Of course, paid proxy server may be permanent. but with these money you can do something better.
IMO the three methods all have disadvantages. and the realization may be none of them. Can anybody tell me the real mechanism of the technique?
Use lots of proxy servers. That will do the trick and since they can be harvested quite easily that's not very hard. Proxy's can be installed on hacked websites for example.
The added question:
Using proxy server. i also think it is impossible to some extent.is there any method to get lots of free proxy servers?
By simply hacking lots of webservers, totally automated, this is possible. For example searching for bad Joomla installs could allow you to install software at each webserver. Also normal computers can be used off course. Like a botnet.
and most free proxy servers is very unstabitily, for there is a possible circumstance that you cannot use the proxy server in one day. Of course, paid proxy server may be permanent. but with these money you can do something better.
Stability is off course important but in this case not really actually. You just send out lots and lots and lots of requests. Don't care which one succeeds and which one doesn't. It doesn't matter for your target.
1. ISP reconnect
This will not work for some (most?) ISPs which will reassign the same IP on a reconnect (as my provider does). Even if it works, you are likely to get the same IP address after some reconnects.
2. IP spoofing
That's the term describing your second method. You change the src-address of the outgoing IP packet. There are two problems with that:
Most ISP's routers don't allow it. They detect that the src address can't come from inside their network, so they simply drop it.
If you have a machine that is allowed to do this (maybe a dedicated server), you can only fake exactly one IP frame. This allows you to, e.g. spoof a DNS request but as you said, you will never get the response. Especially you cannot establish a connection within a stateful protocol like TCP, because this requires a bidirectional handshake. So you can't, e.g., fake a HTTP request using this (even if you don't need the answer)
Proxying
This is the only method that works. You have several options here:
Use open proxy servers (can be found using a search engine, although some will identify themselves as proxies and provide the original IP in the X-Forwarded-For HTTP header, which makes them basically useless for this use case)
Use hacked servers/desktop machines as proxies (maybe from a botnet)
Use free networks like JAP or TOR (the latter of which is probably your best bet, because you can change the exit nodes using some trickery)
If you are going to do something illegal, you might as well go all the way in. There ARE people who run "botnets" which are basically just armies of a few hundred to a few thousand indfected computers (that's what most viruses do). The people who run these armies, actually can charge people a certain amount of money for their "slaves" to visit a website for you (and rate/vote whatever) so you get a few hundred or a few thousand more ratings...
I can't exactly tell where or how much these services cost, since I haven't done it myself, but I know for sure that people over at "H#ckf0rums.net" will do it for you.
If I just want to know if a domain name is reserved; is it sufficient to use this command and see if any domain name servers turn up, in which case it's reserved?
host -t NS example.com
It's a lot faster than visiting http://www.internic.net/whois.html and typing example.com to get much more detailed results, which I'm not interested in anyway.
Absolutely not.
A past employer registered theirname.biz solely for use on the internal network: it had DNS entries on the inward-facing network DNS server, but nowhere on the internet.
I'm not sure the trick was particularly essential, but "imap.theirname.biz" has the advantage over just "imap" that it's unambiguous if you're connected simultaneously to multiple networks (in the absence of deliberate foul play, of course), so you can just use all their internal DNS resolvers. Also the advantage over "imap.theirname.com" that once you know the convention, it's immediately obvious that it's a private server, and hence the reason you can't connect to it is that you forgot to connect VPN. There may have been other benefits to which I was not privy: I'm a coder, not an IT tech...
Various TLDs have differing requirements for whether name servers are provisioned or not. For example ".de" does require that name servers are up and running and correctly configured before they'll allow the domain registration to proceed.
The technical standards for DNS don't require it though, in fact there's nothing in the core DNS specifications to link together the registration of a name with its subsequent operation in the DNS.
Therefore, using whois is probably the most reliable method, with the caveat that you'll need a whois client that's clever enough to figure out which server to talk to for the domain in question.
That said, checking for the appropriate NS record is a very good shortcut to check that a domain is registered, you just can't use the absence of such a record to prove that it isn't!
NS records are not necessarily required for registered domains. The whois service is your most reliable option.
Note that most Unix systems and Mac OS X have a "whois" command line program that is really quick to use:
whois stackoverflow.com
I don't believe that you have to have a DNS pointing to your domain. Even if you had to have a DNS set up, there is no assurance that the box acting as the DNS server isn't down.