How does Tumblr approach custom domain mapping? - dns

I searched all over but can't find a clear answer or even an engineering blog post to describe how companies map custom domains to their application.
For example, let's say I have a Tumblr page with a URL of www.ashley.tumblr.com. The site allows you to add a custom domain so that visiting www.Ashley.com will render www.ashley.tumblr.com with full support for additional pages and directories.
What is the technical name for developing this?

There's no single name for what they're doing - which is engineering their HTTP/web-server code to handle requests from arbitrary HTTP request Host: header and mapping them to their existing Tumblr accounts. It has nothing to do with DNS other than requiring the owner of a custom domain-name to change their A, AAAA, or CNAME records to point to the same host as the non-custom domain (to guarantee this happens it's usual to make the custom domain-name a CNAME for the non-custom domain, in case the non-custom domain's IP address is subject to change).
Exposition time! - Most conventional web-servers (Apache, IIS) are built around the concept of a "website": a physical directory mapped to requests corresponding to a predefined list of HTTP Host: header values (or some wildcard matching pattern) and protocol and port bindings. For example, you'd add an entry called "MyWebsite.com" (the Website Name) that accepts requests to mywebsite.com and www.mywebsite.com (as these are two distinct Host: header values) and maybe some more, like secure.mywebsite.com using HTTPS on port 443).
More modern lightweight webservers and reverse-proxies (like nginx and Node.js' Express) dispense with physical directory mapping and let the application code entirely decide how to route requests within the application's logic (this is what a "router" and/or "demultiplexer" (demux) does in web-application terminology) - this comes at the expense of needing to handle all that logic yourself (to be fair, these webservers come with the necessary tools to easily configure them like the older conventional web-servers, it just isn't the default).
...but the advantage is that you can make it work exactly like you want.
In pseudocode their program probably looks something like this:
void handleRequest(Request request) {
String hostHeader = request.getHeader("Host")
RegexMatch nonCustomDomainMatch = hostHeader.match( "([^\.]+).tumblr.com" )
if nonCustomDomainMatch.success {
String accountName = nonCustomDomainMatch.groups[0]
showAccount( accountName )
}
else {
// Look up the custom domain name in a database or other mutable data store:
String accountName = db.execQuery( "SELECT accountName FROM accounts WHERE accounts.customDomainName = #cdn", new { cdn: hostHeader } )
if accountName == null {
showHttp404Error()
}
else {
showAccount( accountName )
}
}
}
In reality, given their size and scale, it would likely be some custom logic inside hardware load-balancers or some other lightweight frontend service - and always with aggressive caching (database lookups are expensive!).

Related

Generic domain part with fixed subdomain using Caddy and auto SSL?

I'd like to setup a Caddy server where the subdomain is static but the domain part is "wildcard", such as "api.*"
From my understanding of Caddy, the wildcard is possible for one part of the full domain (*.domain.com matches bar.domain.com but not foo.bar.domain.com).
Moreover, this configuration would automatically create a SSL certificates (which Caddy does in general, but I'm not sure here) for any new DNS entry that points to my server with a domain starting with "api.*".
The "*" here would be the domain directly, not any subdomain (it would work for api.domain.com, but not for api.foo.domain.com).
Is this something possible using a simple Caddy command (such as api.* { ... }, which I tried without luck), or does it need a more complex implementation?
Thank you for your help!
I found a working solution with the help of the Caddy Community.
Here's the code :
{
on_demand_tls {
ask https://static.site.com/domain/verify
interval 2m
burst 5
}
}
static.site.com {
...
}
:443 {
tls {
on_demand
}
// Your custom config, for instance:
reverse_proxy * ...
}
The nifty part is the tls { on_demand } part for your generic HTTPS, which will create a certificate automatically. But, this can be abused by anyone that points one of their DNS entry to your server.
So to avoid that, the Caddy community highly recommends you to set a on_demand_tls that will query an endpoint, and allow the SSL certificate to be created only if that endpoint returns true.
NOTE: The ask is a GET request that DO NOT FOLLOW redirects! Anything but a 200 status code will be considered a failure, even a 3xx!
The ask url will have the ?domain appended and will allow you to verify that domain against your logic, such as custom value in the domain like "starting by static.*", and verify that the domain exists in your database (for example).
If your URL already contains some query parameter, don't worry, Caddy is clever enough to add them. (https://static.site.com/domain/verify?some=query will become https://static.site.com/domain/verify?some=query&domain={domain}.
Caddy support https for the ask parameter, and that URL can also be external with no problems at all (no need for localhost or local server configuration).
I met the same problem, and after 1 day's stucking, here is my solution:
Assuming the site name is: site.com, and I want caddy handle these domains for me:
a.dot.site.com
b.dot.site.com
c.dot.site.com
a.eth.site.com
b.eth.site.com
c.eth.site.com
1.make sure you set SSL access available. e.g. via cloudflare:
2.set the A address pointing to your Caddy server's IP.
2.Caddy file should looks like:
# the key is: you have to list all the patterns for your multiple subdomains
*.site.com *.eth.site.com *.dot.site.com {
reverse_proxy 127.0.0.1:4567
log {
output file /var/log/access-wildcard-site.com.log
}
tls {
dns cloudflare <your cloud flare api key>
}
}

Azure Application Gateway Redirection from empty hostname

I have created an Application Gateway that needs to fulfill the working of my previous Resource (F5).
As a listener I use a hostname: hostname.stackoverflow.com that listens on 443
As a Http Setting I am using a specific port being 4443
As a BackEnd pool I use the URL/FQDN of my dev VM.
This totally works If i create a VM in the VNET and add "hostname.stackoverflow.com" to the hosts file with the ip of the application gateway.
Now I want to get a little further and add paths to my Application Gateway.
The goal is that if I use "hostname.stackoverflow.com" I need to redirect this to "Hostname.stackoverflow.com/login.aspx?guestlogin".
As far I have tried the following.
Add the "/login.aspx?guestLogin" to the HTTPS settings like this.
When I try this inside my VM. The URL changes but the path that I added there was not added in the right way, This is what I got:
So That made me think override backend path is maybe not the right way to do this.
Wanted To create a Redirection Rule That will redirect my "hostname.stackoverflow.com" to the "hostname.stackoverflow.com/login.aspx?guestLogin" But in the settings of the Application Gateway I need to provide a source path (meaning: I can not redirect from an empty hostname to a new url I think)
I am very new to Azure and even more new to the Application Gateway. Is there something that I did wrong. Is there a better way to do this ?
The iRule that I need to get in Application Gateway is as followed.
if { [string tolower [HTTP::host]] equals "hostname.stackoverflow.com" } {
if {[HTTP::path] eq "/"} {
HTTP::redirect "login.aspx?guestLogin"
}
elseif {[string tolower [HTTP::uri]] starts_with "/login.aspx?id="} {
set tail [string range [HTTP::uri] 12 end]
HTTP::redirect "login.aspx?guestLogin&$tail"
}
pool default.pool
}

Creating more than one site in Kentico?

Am going to use Kentico to create more than one store (Site) and assign user for each store to add/modify/delete his products, i've created 2 stores the first one with domain localhost:8080 and second one is storeone.localhost:8080 as documentation said in Kentico Doc URL, i can open first site with no problem but when i tried to switch to second Site it gives me Bad Request - Invalid Hostname .. can any one help me in this?? .. also i would appreciate it if any one help me on how to extract product data using Kentico API's as documentation provide me only with updating/modifying/removing data from database and i want to know how to display it with it's attachments like images pdf that i've uploaded it.
The best approach is to use two different ports. The reason for this is IIS is by default bound to port 80. So what I'd do is leave one site at 80 and do another at say 2. Make these bindings in IIS then go to Kentico and add your second site at localhost:2 vs. :8080. There's a conflict with port numbers. Kentico and IIS are "confused" and don't know which one to serve up. The only way it will work with the same port is to start and stop sites within Kentico.
Brenden is correct - there cannot be 2 sites running on same domain. What you need to do is configure IIS bindings. What I often do is that I configure my hosts file (C:\Windows\System32\drivers\etc) and add a few more rules like:
127.0.0.1 localhost2
127.0.0.1 localhost3
And then I can use bind my Kentico sites to these domains. Don't forget to also change the domain names in Kentico -> Sites app.
As for your second question:
It depends whether you want to get only SKUInfo object or page object where the custom data (page type fields) are stored. If you just need SKUInfo you can use something like:
// gets only corresponding SKU Info object
var singleProduct = SKUInfoProvider.GetSKUInfo(1); // SKUID from COM_SKU table
if (singleProduct != null)
{
var name = singleProduct.SKUName;
var price = singleProduct.SKUPrice;
}
If you need to get the product with all custom fields you need to use the Pages API as you would with any other page. A simple example:
// gets sku with all custom properties
var tree = new TreeProvider(MembershipContext.AuthenticatedUser);
var singleProduct = tree.SelectSingleDocument(2); // DocumentID from CMS_Document table
if (singleProduct != null)
{
// work with product
}
// or for multiple products
var products = tree.SelectNodes("custom.myProductType");
foreach (var product in products)
{
// work with products/pages
}
For the purpose of retrieving pages I would highly recommend to check this documentation article which contains a lot of examples.

Why should i use a LoadBalancerProbe instead of subscribing to the RoleEnvironment.StatusCheck Event?

I was fiddling with the options azure provides to balance the load between multiple webroles.
I found three possible ways to do this.
the first would be to do nothing at all and let the default ( round robin) implementation do the job.
the second possibility would be to define a custom LoadBalancerProbe in the ServiceDefinitionFile, which i tried and did not get to work: From my understanding the custom aspx page is called each time a status check is performed on the role. Depending on the http response code the role changes its status to busy. - but this is never happening.
Also, i couldnt really find any examples for defining a custom LoadBalancingProbe.
Thus i looked for an alternative way to do this.
Now iam subscribing to the RoleEnvironment.StatusCheck Event, which allows me to implement some logic and depending on the results set the role state to busy and available.
My Questions:
1) Supposing the Custom LoadBalancerProbe works as described in the MSDN, what is the difference between subscribing to the StatusCheckEvent and using a custom probe?
2) Why does my custom load balancer probe not work ? - iam just testing with the azure emulator for now and iam well aware that traffic still gets routed to the webrole instances allthough they are set to busy in the emulator.
But my Custom Probe does not change the status of the webroleinstances at all.
Here is the very rudimentary code, which should - to my knowledge set the status of webrole instance_n_0 to busy.
public class LoadBalanceController : Controller
{
public ActionResult Index()
{
WebOperationContext woc = WebOperationContext.Current;
if(RoleEnvironment.CurrentRoleInstance.Id.ToLower().Contains("_0"))
{
woc.OutgoingResponse.StatusCode = System.Net.HttpStatusCode.ServiceUnavailable;
}else
{
woc.OutgoingResponse.StatusCode = System.Net.HttpStatusCode.OK;
}
return View(); //not relevant
}
Ive also configered my servicedefinitionfile and set a Route to redirect to this controller/action when calling the healthcheck.aspx defined in the custom probe.
<LoadBalancerProbes>
<LoadBalancerProbe name="WebRoleBalancerProbeHttp" protocol="http" path="healthcheck.aspx" intervalInSeconds="5" timeoutInSeconds="100"/>
</LoadBalancerProbes>
...
<InputEndpoint name="EndpointWeb" protocol="http" port="80" loadBalancerProbe="WebRoleBalancerProbeHttp"/>
The Route:
routes.MapRoute(
name: "HealhCheck",
url: "healthcheck.aspx",
defaults: new { controller = "LoadBalance", action = "Index", id = UrlParameter.Optional }
);
Not sure why the custom probe isn't working, but for the differences: The health-check event lets you announce whether an instance is available, but you don't have any flexibility in terms of how often this is called. Also, you can't launch a separate service that listens on a custom port (or port type).
You have much more flexibility with custom probes, since you can create any type of port listener to determine health, even a separate exe.
With Virtual Machines, this is the only method of health probes, since Virtual Machines don't have the guest agent running and don't provide the health-check event.
In your service definition file, you have marked your probe to be hosted on an http endpoint (and not HTTPS).
Is that the case with your web app as well ? Is it exposing that endpoint on HTTP and not on HTTPS ? If yes, then also check if there is any automatic redirection to HTTPS happening.
I think you have mostly setup everything properly. Here is a post that has some example code and another question from SO, where they were successful in setting it up (should provide some insight into the csdef file).
I agree with David Makogon's points about the differences between the two.

Is there a way to get the IP address of the client/caller in a Shiro Filter in Grails

sorry if this is a dumb question. I have some web service calls that are implemented in Grails controllers and we use the Shiro plugin for security. I want to be able to create a whitelist of IP addresses for certain operations that should only come from our own servers or trusted partners. I am finding the documentation a little spares on this subject. My first thought was to try and implement the whitelist here. I wouldn't be surprised if there is a simpler way to do this. I am a bit of a Shiro newb. Could sure use a copy of Shiro for Dummies!
class ShiroSecurityFilters {
def filters = {
all(uri: "/**") {
before = {
// Ignore direct views (e.g. the default main index page).
if (controllerName in ['foo', 'bar']) {
return true
}
// Access control by convention.
accessControl()
}
}
}
}
You should have access to the request object in the Filter and you can call request.getRemoteAddr() to access the IP address.

Resources